2023-04-03 02:53:34,487 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-04-03 02:53:34,763 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c 2023-04-03 02:53:34,776 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-04-03 02:53:34,777 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc, deleteOnExit=true 2023-04-03 02:53:34,777 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-04-03 02:53:34,778 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/test.cache.data in system properties and HBase conf 2023-04-03 02:53:34,778 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/hadoop.tmp.dir in system properties and HBase conf 2023-04-03 02:53:34,778 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/hadoop.log.dir in system properties and HBase conf 2023-04-03 02:53:34,779 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/mapreduce.cluster.local.dir in system properties and HBase conf 2023-04-03 02:53:34,779 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-04-03 02:53:34,780 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-04-03 02:53:34,895 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-04-03 02:53:35,331 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-04-03 02:53:35,335 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-04-03 02:53:35,335 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-04-03 02:53:35,336 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-04-03 02:53:35,336 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-03 02:53:35,336 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-04-03 02:53:35,337 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-04-03 02:53:35,337 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-03 02:53:35,337 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-03 02:53:35,338 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-04-03 02:53:35,338 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/nfs.dump.dir in system properties and HBase conf 2023-04-03 02:53:35,338 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/java.io.tmpdir in system properties and HBase conf 2023-04-03 02:53:35,339 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-03 02:53:35,339 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-04-03 02:53:35,339 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-04-03 02:53:35,978 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-03 02:53:35,982 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-03 02:53:36,261 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-04-03 02:53:36,442 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-04-03 02:53:36,466 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-03 02:53:36,505 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-04-03 02:53:36,538 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/java.io.tmpdir/Jetty_localhost_37685_hdfs____.5an4hz/webapp 2023-04-03 02:53:36,688 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37685 2023-04-03 02:53:36,704 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-03 02:53:36,705 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-03 02:53:37,287 WARN [Listener at localhost/41571] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-03 02:53:37,363 WARN [Listener at localhost/41571] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-03 02:53:37,382 WARN [Listener at localhost/41571] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-03 02:53:37,389 INFO [Listener at localhost/41571] log.Slf4jLog(67): jetty-6.1.26 2023-04-03 02:53:37,394 INFO [Listener at localhost/41571] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/java.io.tmpdir/Jetty_localhost_34877_datanode____.72yema/webapp 2023-04-03 02:53:37,503 INFO [Listener at localhost/41571] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34877 2023-04-03 02:53:37,856 WARN [Listener at localhost/37595] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-03 02:53:38,363 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x40b5d94aee4473cd: Processing first storage report for DS-70f75d64-e46c-43cb-b1f9-855efa14e91c from datanode 1818b857-e78f-4c89-92f7-68f8617e3cc1 2023-04-03 02:53:38,364 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x40b5d94aee4473cd: from storage DS-70f75d64-e46c-43cb-b1f9-855efa14e91c node DatanodeRegistration(127.0.0.1:33503, datanodeUuid=1818b857-e78f-4c89-92f7-68f8617e3cc1, infoPort=33809, infoSecurePort=0, ipcPort=37595, storageInfo=lv=-57;cid=testClusterID;nsid=1294390512;c=1680490416057), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-04-03 02:53:38,364 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x40b5d94aee4473cd: Processing first storage report for DS-4b12322c-35ba-4d7d-821e-0fba7d89a10c from datanode 1818b857-e78f-4c89-92f7-68f8617e3cc1 2023-04-03 02:53:38,364 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x40b5d94aee4473cd: from storage DS-4b12322c-35ba-4d7d-821e-0fba7d89a10c node DatanodeRegistration(127.0.0.1:33503, datanodeUuid=1818b857-e78f-4c89-92f7-68f8617e3cc1, infoPort=33809, infoSecurePort=0, ipcPort=37595, storageInfo=lv=-57;cid=testClusterID;nsid=1294390512;c=1680490416057), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-03 02:53:38,428 DEBUG [Listener at localhost/37595] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c 2023-04-03 02:53:38,603 INFO [Listener at localhost/37595] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/zookeeper_0, clientPort=52880, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-04-03 02:53:38,616 INFO [Listener at localhost/37595] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52880 2023-04-03 02:53:38,623 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:38,625 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:39,348 INFO [Listener at localhost/37595] util.FSUtils(471): Created version file at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00 with version=8 2023-04-03 02:53:39,349 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/hbase-staging 2023-04-03 02:53:39,672 INFO [Listener at localhost/37595] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-04-03 02:53:40,139 INFO [Listener at localhost/37595] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-04-03 02:53:40,172 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,173 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,173 INFO [Listener at localhost/37595] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-03 02:53:40,173 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,173 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-03 02:53:40,317 INFO [Listener at localhost/37595] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-04-03 02:53:40,388 DEBUG [Listener at localhost/37595] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-04-03 02:53:40,481 INFO [Listener at localhost/37595] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34259 2023-04-03 02:53:40,491 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:40,493 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:40,514 INFO [Listener at localhost/37595] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34259 connecting to ZooKeeper ensemble=127.0.0.1:52880 2023-04-03 02:53:40,555 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:342590x0, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-03 02:53:40,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:34259-0x10147f0bb620000 connected 2023-04-03 02:53:40,605 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-03 02:53:40,606 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-03 02:53:40,609 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-03 02:53:40,617 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34259 2023-04-03 02:53:40,618 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34259 2023-04-03 02:53:40,618 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34259 2023-04-03 02:53:40,620 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34259 2023-04-03 02:53:40,622 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34259 2023-04-03 02:53:40,629 INFO [Listener at localhost/37595] master.HMaster(444): hbase.rootdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00, hbase.cluster.distributed=false 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-03 02:53:40,720 INFO [Listener at localhost/37595] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-03 02:53:40,725 INFO [Listener at localhost/37595] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-03 02:53:40,728 INFO [Listener at localhost/37595] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45927 2023-04-03 02:53:40,730 INFO [Listener at localhost/37595] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-03 02:53:40,736 DEBUG [Listener at localhost/37595] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-03 02:53:40,737 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:40,740 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:40,741 INFO [Listener at localhost/37595] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45927 connecting to ZooKeeper ensemble=127.0.0.1:52880 2023-04-03 02:53:40,745 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:459270x0, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-03 02:53:40,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45927-0x10147f0bb620001 connected 2023-04-03 02:53:40,746 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-03 02:53:40,747 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-03 02:53:40,747 DEBUG [Listener at localhost/37595] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-03 02:53:40,752 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45927 2023-04-03 02:53:40,752 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45927 2023-04-03 02:53:40,752 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45927 2023-04-03 02:53:40,753 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45927 2023-04-03 02:53:40,753 DEBUG [Listener at localhost/37595] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45927 2023-04-03 02:53:40,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:40,765 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-03 02:53:40,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:40,784 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-03 02:53:40,784 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-03 02:53:40,784 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:40,785 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-03 02:53:40,786 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,34259,1680490419507 from backup master directory 2023-04-03 02:53:40,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-03 02:53:40,789 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:40,789 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-03 02:53:40,790 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-03 02:53:40,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:40,793 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-04-03 02:53:40,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-04-03 02:53:40,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/hbase.id with ID: c610d160-0b13-4009-8e88-10e179e74ec3 2023-04-03 02:53:40,921 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:40,936 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:40,981 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x7dd79577 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:41,026 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2960c227, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:41,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-03 02:53:41,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-04-03 02:53:41,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-04-03 02:53:41,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-04-03 02:53:41,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-03 02:53:41,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-03 02:53:41,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-03 02:53:41,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store-tmp 2023-04-03 02:53:41,142 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:41,142 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-03 02:53:41,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:53:41,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:53:41,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-03 02:53:41,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:53:41,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:53:41,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-03 02:53:41,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/WALs/jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:41,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34259%2C1680490419507, suffix=, logDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/WALs/jenkins-hbase4.apache.org,34259,1680490419507, archiveDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/oldWALs, maxLogs=10 2023-04-03 02:53:41,220 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK] 2023-04-03 02:53:41,228 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-04-03 02:53:41,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/WALs/jenkins-hbase4.apache.org,34259,1680490419507/jenkins-hbase4.apache.org%2C34259%2C1680490419507.1680490421177 2023-04-03 02:53:41,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK]] 2023-04-03 02:53:41,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:41,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:41,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,300 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,362 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,370 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-04-03 02:53:41,397 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-04-03 02:53:41,411 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:41,418 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-04-03 02:53:41,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:53:41,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67596611, jitterRate=0.007267996668815613}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:53:41,443 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-03 02:53:41,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-04-03 02:53:41,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-04-03 02:53:41,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-04-03 02:53:41,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-03 02:53:41,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-03 02:53:41,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-04-03 02:53:41,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-04-03 02:53:41,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-04-03 02:53:41,541 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-04-03 02:53:41,567 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-04-03 02:53:41,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-04-03 02:53:41,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-04-03 02:53:41,578 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-04-03 02:53:41,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-04-03 02:53:41,585 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:41,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-04-03 02:53:41,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-04-03 02:53:41,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-04-03 02:53:41,603 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-03 02:53:41,603 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-03 02:53:41,603 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:41,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,34259,1680490419507, sessionid=0x10147f0bb620000, setting cluster-up flag (Was=false) 2023-04-03 02:53:41,620 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:41,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-04-03 02:53:41,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:41,633 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:41,639 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-04-03 02:53:41,640 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:41,643 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp 2023-04-03 02:53:41,657 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(952): ClusterId : c610d160-0b13-4009-8e88-10e179e74ec3 2023-04-03 02:53:41,663 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-03 02:53:41,669 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-03 02:53:41,669 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-03 02:53:41,674 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-03 02:53:41,675 DEBUG [RS:0;jenkins-hbase4:45927] zookeeper.ReadOnlyZKClient(139): Connect 0x502f0474 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:41,680 DEBUG [RS:0;jenkins-hbase4:45927] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ee8e001, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:41,680 DEBUG [RS:0;jenkins-hbase4:45927] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ad76f04, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-04-03 02:53:41,713 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45927 2023-04-03 02:53:41,718 INFO [RS:0;jenkins-hbase4:45927] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-03 02:53:41,718 INFO [RS:0;jenkins-hbase4:45927] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-03 02:53:41,718 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1023): About to register with Master. 2023-04-03 02:53:41,722 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,34259,1680490419507 with isa=jenkins-hbase4.apache.org/172.31.14.131:45927, startcode=1680490420719 2023-04-03 02:53:41,744 DEBUG [RS:0;jenkins-hbase4:45927] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-03 02:53:41,771 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:41,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-04-03 02:53:41,781 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:41,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680490451783 2023-04-03 02:53:41,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-04-03 02:53:41,788 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-04-03 02:53:41,789 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-04-03 02:53:41,797 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-03 02:53:41,798 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-04-03 02:53:41,805 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58537, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-04-03 02:53:41,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-04-03 02:53:41,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-04-03 02:53:41,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-04-03 02:53:41,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-04-03 02:53:41,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:41,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-04-03 02:53:41,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-04-03 02:53:41,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-04-03 02:53:41,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-04-03 02:53:41,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-04-03 02:53:41,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-04-03 02:53:41,819 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1680490421819,5,FailOnTimeoutGroup] 2023-04-03 02:53:41,821 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1680490421819,5,FailOnTimeoutGroup] 2023-04-03 02:53:41,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:41,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-04-03 02:53:41,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:41,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:41,848 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(2829): Master is not running yet 2023-04-03 02:53:41,848 WARN [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-04-03 02:53:41,949 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,34259,1680490419507 with isa=jenkins-hbase4.apache.org/172.31.14.131:45927, startcode=1680490420719 2023-04-03 02:53:41,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:41,964 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00 2023-04-03 02:53:41,965 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41571 2023-04-03 02:53:41,965 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-03 02:53:41,971 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-03 02:53:41,972 DEBUG [RS:0;jenkins-hbase4:45927] zookeeper.ZKUtil(162): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:41,972 WARN [RS:0;jenkins-hbase4:45927] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-03 02:53:41,972 INFO [RS:0;jenkins-hbase4:45927] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-03 02:53:41,973 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:41,974 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45927,1680490420719] 2023-04-03 02:53:41,984 DEBUG [RS:0;jenkins-hbase4:45927] zookeeper.ZKUtil(162): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:41,995 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-03 02:53:42,005 INFO [RS:0;jenkins-hbase4:45927] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-03 02:53:42,025 INFO [RS:0;jenkins-hbase4:45927] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-03 02:53:42,028 INFO [RS:0;jenkins-hbase4:45927] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:53:42,028 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,030 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-03 02:53:42,037 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,037 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,037 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,038 DEBUG [RS:0;jenkins-hbase4:45927] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-03 02:53:42,039 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,039 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,040 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,057 INFO [RS:0;jenkins-hbase4:45927] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-03 02:53:42,060 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45927,1680490420719-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:42,077 INFO [RS:0;jenkins-hbase4:45927] regionserver.Replication(203): jenkins-hbase4.apache.org,45927,1680490420719 started 2023-04-03 02:53:42,077 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,45927,1680490420719, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45927, sessionid=0x10147f0bb620001 2023-04-03 02:53:42,077 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-03 02:53:42,077 DEBUG [RS:0;jenkins-hbase4:45927] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:42,077 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45927,1680490420719' 2023-04-03 02:53:42,077 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:42,078 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45927,1680490420719' 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-03 02:53:42,079 DEBUG [RS:0;jenkins-hbase4:45927] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-03 02:53:42,080 DEBUG [RS:0;jenkins-hbase4:45927] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-03 02:53:42,080 INFO [RS:0;jenkins-hbase4:45927] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-03 02:53:42,080 INFO [RS:0;jenkins-hbase4:45927] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-03 02:53:42,191 INFO [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45927%2C1680490420719, suffix=, logDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719, archiveDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/oldWALs, maxLogs=32 2023-04-03 02:53:42,210 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK] 2023-04-03 02:53:42,214 INFO [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719/jenkins-hbase4.apache.org%2C45927%2C1680490420719.1680490422193 2023-04-03 02:53:42,214 DEBUG [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK]] 2023-04-03 02:53:42,236 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:42,238 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:42,238 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00 2023-04-03 02:53:42,657 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:42,660 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-03 02:53:42,662 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/info 2023-04-03 02:53:42,663 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-03 02:53:42,664 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:42,664 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-03 02:53:42,667 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/rep_barrier 2023-04-03 02:53:42,667 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-03 02:53:42,668 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:42,668 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-03 02:53:42,670 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/table 2023-04-03 02:53:42,671 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-03 02:53:42,672 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:42,673 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740 2023-04-03 02:53:42,674 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740 2023-04-03 02:53:42,677 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-03 02:53:42,679 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-03 02:53:42,682 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:53:42,683 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65393017, jitterRate=-0.02556811273097992}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-03 02:53:42,683 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-03 02:53:42,683 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-03 02:53:42,683 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-03 02:53:42,683 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-03 02:53:42,683 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-03 02:53:42,683 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-03 02:53:42,684 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-03 02:53:42,684 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-03 02:53:42,688 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-04-03 02:53:42,688 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-04-03 02:53:42,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-04-03 02:53:42,707 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-04-03 02:53:42,710 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-03 02:53:42,861 DEBUG [jenkins-hbase4:34259] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-04-03 02:53:42,864 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45927,1680490420719, state=OPENING 2023-04-03 02:53:42,871 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-04-03 02:53:42,874 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:42,875 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-03 02:53:42,877 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:43,062 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:43,065 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-03 02:53:43,068 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36204, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-03 02:53:43,079 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-04-03 02:53:43,079 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-03 02:53:43,083 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45927%2C1680490420719.meta, suffix=.meta, logDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719, archiveDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/oldWALs, maxLogs=32 2023-04-03 02:53:43,099 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK] 2023-04-03 02:53:43,103 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719/jenkins-hbase4.apache.org%2C45927%2C1680490420719.meta.1680490423084.meta 2023-04-03 02:53:43,104 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33503,DS-70f75d64-e46c-43cb-b1f9-855efa14e91c,DISK]] 2023-04-03 02:53:43,104 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:43,106 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-04-03 02:53:43,127 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-04-03 02:53:43,132 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-04-03 02:53:43,138 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-04-03 02:53:43,138 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:43,138 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-04-03 02:53:43,138 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-04-03 02:53:43,141 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-03 02:53:43,143 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/info 2023-04-03 02:53:43,143 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/info 2023-04-03 02:53:43,143 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-03 02:53:43,144 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:43,145 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-03 02:53:43,146 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/rep_barrier 2023-04-03 02:53:43,146 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/rep_barrier 2023-04-03 02:53:43,147 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-03 02:53:43,148 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:43,148 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-03 02:53:43,149 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/table 2023-04-03 02:53:43,149 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/table 2023-04-03 02:53:43,150 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-03 02:53:43,151 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:43,152 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740 2023-04-03 02:53:43,155 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740 2023-04-03 02:53:43,158 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-03 02:53:43,160 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-03 02:53:43,162 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73274132, jitterRate=0.09186965227127075}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-03 02:53:43,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-03 02:53:43,171 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1680490423054 2023-04-03 02:53:43,187 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-04-03 02:53:43,188 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-04-03 02:53:43,188 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45927,1680490420719, state=OPEN 2023-04-03 02:53:43,191 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-04-03 02:53:43,191 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-03 02:53:43,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-04-03 02:53:43,195 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45927,1680490420719 in 314 msec 2023-04-03 02:53:43,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-04-03 02:53:43,202 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 499 msec 2023-04-03 02:53:43,207 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5140 sec 2023-04-03 02:53:43,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1680490423207, completionTime=-1 2023-04-03 02:53:43,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-04-03 02:53:43,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-04-03 02:53:43,269 DEBUG [hconnection-0x9c0afc6-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:53:43,273 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36210, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:53:43,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-04-03 02:53:43,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680490483292 2023-04-03 02:53:43,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680490543292 2023-04-03 02:53:43,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 84 msec 2023-04-03 02:53:43,320 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34259,1680490419507-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:43,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34259,1680490419507-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:43,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34259,1680490419507-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:43,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:34259, period=300000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:43,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-04-03 02:53:43,330 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-04-03 02:53:43,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-04-03 02:53:43,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-03 02:53:43,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-04-03 02:53:43,352 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:53:43,355 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:53:43,378 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:43,381 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49 empty. 2023-04-03 02:53:43,382 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:43,382 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-04-03 02:53:43,808 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:43,810 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4d156a638f44baaa58723e6611f8ec49, NAME => 'hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:53:44,222 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:44,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 4d156a638f44baaa58723e6611f8ec49, disabling compactions & flushes 2023-04-03 02:53:44,223 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. after waiting 0 ms 2023-04-03 02:53:44,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,223 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,223 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 4d156a638f44baaa58723e6611f8ec49: 2023-04-03 02:53:44,228 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:53:44,242 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680490424230"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490424230"}]},"ts":"1680490424230"} 2023-04-03 02:53:44,267 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:53:44,269 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:53:44,274 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490424270"}]},"ts":"1680490424270"} 2023-04-03 02:53:44,278 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-04-03 02:53:44,288 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4d156a638f44baaa58723e6611f8ec49, ASSIGN}] 2023-04-03 02:53:44,291 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4d156a638f44baaa58723e6611f8ec49, ASSIGN 2023-04-03 02:53:44,292 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=4d156a638f44baaa58723e6611f8ec49, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:53:44,443 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4d156a638f44baaa58723e6611f8ec49, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:44,444 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680490424443"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490424443"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490424443"}]},"ts":"1680490424443"} 2023-04-03 02:53:44,448 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 4d156a638f44baaa58723e6611f8ec49, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:44,609 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,610 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4d156a638f44baaa58723e6611f8ec49, NAME => 'hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:44,610 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,611 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:44,611 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,611 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,613 INFO [StoreOpener-4d156a638f44baaa58723e6611f8ec49-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,615 DEBUG [StoreOpener-4d156a638f44baaa58723e6611f8ec49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/info 2023-04-03 02:53:44,615 DEBUG [StoreOpener-4d156a638f44baaa58723e6611f8ec49-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/info 2023-04-03 02:53:44,616 INFO [StoreOpener-4d156a638f44baaa58723e6611f8ec49-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4d156a638f44baaa58723e6611f8ec49 columnFamilyName info 2023-04-03 02:53:44,617 INFO [StoreOpener-4d156a638f44baaa58723e6611f8ec49-1] regionserver.HStore(310): Store=4d156a638f44baaa58723e6611f8ec49/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:44,619 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,620 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,625 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:53:44,629 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:53:44,630 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4d156a638f44baaa58723e6611f8ec49; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74262116, jitterRate=0.10659176111221313}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:53:44,630 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4d156a638f44baaa58723e6611f8ec49: 2023-04-03 02:53:44,632 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49., pid=6, masterSystemTime=1680490424602 2023-04-03 02:53:44,636 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,636 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:53:44,637 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4d156a638f44baaa58723e6611f8ec49, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:44,637 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680490424636"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490424636"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490424636"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490424636"}]},"ts":"1680490424636"} 2023-04-03 02:53:44,645 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-04-03 02:53:44,645 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 4d156a638f44baaa58723e6611f8ec49, server=jenkins-hbase4.apache.org,45927,1680490420719 in 193 msec 2023-04-03 02:53:44,649 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-04-03 02:53:44,650 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=4d156a638f44baaa58723e6611f8ec49, ASSIGN in 357 msec 2023-04-03 02:53:44,651 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:53:44,652 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490424652"}]},"ts":"1680490424652"} 2023-04-03 02:53:44,655 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-04-03 02:53:44,659 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:53:44,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3190 sec 2023-04-03 02:53:44,753 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-04-03 02:53:44,754 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-04-03 02:53:44,754 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:53:44,787 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-04-03 02:53:44,800 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-03 02:53:44,807 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 27 msec 2023-04-03 02:53:44,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-04-03 02:53:44,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-03 02:53:44,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-04-03 02:53:44,847 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-04-03 02:53:44,849 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-04-03 02:53:44,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.060sec 2023-04-03 02:53:44,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-04-03 02:53:44,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-04-03 02:53:44,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-04-03 02:53:44,854 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34259,1680490419507-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-04-03 02:53:44,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34259,1680490419507-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-04-03 02:53:44,864 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(139): Connect 0x0672b468 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:44,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-04-03 02:53:44,869 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@561bf69b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:44,883 DEBUG [hconnection-0x78f916d2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:53:44,894 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60172, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:53:44,904 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:53:44,905 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(139): Connect 0x33fc1e18 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:44,909 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b586bb5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:44,910 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:53:44,932 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=442, ProcessCount=170, AvailableMemoryMB=4328 2023-04-03 02:53:44,942 DEBUG [Listener at localhost/37595] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:53:44,946 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34756, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:53:44,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:53:44,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:44,961 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:53:44,963 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:53:44,966 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:44,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-04-03 02:53:44,967 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 empty. 2023-04-03 02:53:44,968 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:44,968 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-04-03 02:53:44,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-03 02:53:45,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-03 02:53:45,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-03 02:53:45,387 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:45,389 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6b7f8c28a8619bf76181b65e5ae70d83, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:53:45,400 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:45,400 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 6b7f8c28a8619bf76181b65e5ae70d83, disabling compactions & flushes 2023-04-03 02:53:45,400 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,400 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,401 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. after waiting 0 ms 2023-04-03 02:53:45,401 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,401 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,401 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:45,405 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:53:45,407 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490425407"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490425407"}]},"ts":"1680490425407"} 2023-04-03 02:53:45,410 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:53:45,411 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:53:45,411 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490425411"}]},"ts":"1680490425411"} 2023-04-03 02:53:45,414 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-04-03 02:53:45,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, ASSIGN}] 2023-04-03 02:53:45,420 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, ASSIGN 2023-04-03 02:53:45,422 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:53:45,573 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:45,573 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490425573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490425573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490425573"}]},"ts":"1680490425573"} 2023-04-03 02:53:45,576 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:45,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-03 02:53:45,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b7f8c28a8619bf76181b65e5ae70d83, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:45,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:45,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,739 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,741 DEBUG [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:45,741 DEBUG [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:45,744 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b7f8c28a8619bf76181b65e5ae70d83 columnFamilyName family1 2023-04-03 02:53:45,748 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] regionserver.HStore(310): Store=6b7f8c28a8619bf76181b65e5ae70d83/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:45,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:45,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:53:45,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6b7f8c28a8619bf76181b65e5ae70d83; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65936479, jitterRate=-0.017469897866249084}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:53:45,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:45,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., pid=11, masterSystemTime=1680490425730 2023-04-03 02:53:45,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:45,766 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:45,766 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490425766"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490425766"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490425766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490425766"}]},"ts":"1680490425766"} 2023-04-03 02:53:45,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-04-03 02:53:45,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719 in 193 msec 2023-04-03 02:53:45,776 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-04-03 02:53:45,776 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, ASSIGN in 354 msec 2023-04-03 02:53:45,777 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:53:45,777 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490425777"}]},"ts":"1680490425777"} 2023-04-03 02:53:45,779 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-04-03 02:53:45,782 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:53:45,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 825 msec 2023-04-03 02:53:46,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-03 02:53:46,091 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-04-03 02:53:46,144 DEBUG [hconnection-0x3666aa34-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:53:46,154 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60182, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:53:46,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:46,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,429 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:46,429 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:46,432 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:46,432 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:46,436 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,436 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,437 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,439 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:46,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,440 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:46,444 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,444 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:46,447 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:46,468 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:46,468 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:46,469 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:46,480 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:46,480 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:46,481 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:46,482 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:46,484 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:53:47,172 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306062b283020a452d87ec117b59257f91 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511306062b283020a452d87ec117b59257f91 2023-04-03 02:53:47,599 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/01b0499e3b784edd8803b49b0b15f17c 2023-04-03 02:53:47,621 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/01b0499e3b784edd8803b49b0b15f17c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c 2023-04-03 02:53:47,637 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c, entries=1000, sequenceid=5, filesize=175.8 K 2023-04-03 02:53:47,640 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 1157ms, sequenceid=5, compaction requested=false 2023-04-03 02:53:47,642 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:47,647 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:47,649 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:47,650 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:47,654 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:47,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:47,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:47,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:47,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:47,657 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:47,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57b76a2e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:47,657 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,659 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:47,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,659 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:47,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:47,659 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:47,662 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,662 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:47,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:47,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:47,662 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:47,662 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:47,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:47,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:47,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:47,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:47,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:47,667 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:47,667 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:47,667 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:47,667 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,667 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:47,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:47,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:47,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:47,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:47,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:47,672 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:47,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,674 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,684 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:47,684 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:47,684 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,684 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,685 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:47,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:47,684 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:47,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:47,685 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:47,685 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,685 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,685 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:47,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:47,687 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:47,688 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:47,788 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:47,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:47,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:47,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:47,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,824 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:47,824 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:47,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:47,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:47,826 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,826 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,827 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:47,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:47,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:47,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,839 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,840 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:47,840 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:47,840 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:47,840 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:47,840 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:47,840 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:47,841 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:47,844 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:47,844 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:47,844 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:47,844 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:47,844 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:53:48,000 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-04-03 02:53:48,001 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-04-03 02:53:48,307 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128608129eb036f463a9e4a3a9bbfd584b0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128608129eb036f463a9e4a3a9bbfd584b0 2023-04-03 02:53:48,346 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/7bcd93380365490fb328928d35c5161c 2023-04-03 02:53:48,369 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/7bcd93380365490fb328928d35c5161c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c 2023-04-03 02:53:48,382 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:53:48,384 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 540ms, sequenceid=9, compaction requested=false 2023-04-03 02:53:48,385 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:48,385 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:48,385 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:48,386 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,389 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:48,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:48,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:48,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:48,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20388fbf[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:48,393 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:48,393 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,394 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:48,395 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:48,395 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:48,395 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,395 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:48,397 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:48,397 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,397 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:48,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:48,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:48,397 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:48,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:48,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:48,401 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:48,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:48,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:48,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,401 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:48,406 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,406 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:48,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:48,406 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:48,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,407 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:48,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:48,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:48,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:48,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,409 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:48,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,413 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:48,413 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:48,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:48,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,414 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:48,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:48,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:48,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:48,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:48,415 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:48,416 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:48,416 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:48,516 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:48,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:48,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:48,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:48,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,533 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:48,533 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:48,534 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:48,534 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:48,535 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,535 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,536 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:48,536 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:48,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:48,536 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:48,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,536 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:48,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:48,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:48,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:48,555 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:48,555 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:48,555 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:48,555 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:48,555 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:48,555 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:48,556 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:48,556 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:48,672 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:53:49,025 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512017cc61831b2954aaea93aa709cd76b21d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512017cc61831b2954aaea93aa709cd76b21d 2023-04-03 02:53:49,448 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/95953b1daf934cef8795705b13797865 2023-04-03 02:53:49,459 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/95953b1daf934cef8795705b13797865 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865 2023-04-03 02:53:49,472 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865, entries=1, sequenceid=13, filesize=5.0 K 2023-04-03 02:53:49,474 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 918ms, sequenceid=13, compaction requested=true 2023-04-03 02:53:49,474 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:49,474 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:49,474 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:49,475 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:49,475 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,475 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:49,475 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:49,479 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,479 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,480 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,480 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:49,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:49,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69b31ef0[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:49,483 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:49,483 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,485 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,485 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:49,485 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:49,485 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:49,485 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,485 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:49,487 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:49,487 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,487 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:49,488 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:49,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:49,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:49,501 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:49,501 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:49,501 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:49,501 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,501 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:49,504 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,504 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,504 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:49,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,506 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,509 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,515 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:49,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:49,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,516 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:49,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:49,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:49,516 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:49,516 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:49,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:49,616 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:49,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:49,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:49,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:49,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,652 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:49,652 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:49,653 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:49,653 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:49,654 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,654 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,655 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:49,655 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:49,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:49,656 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,656 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:49,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:49,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,657 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,659 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:49,659 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,659 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:49,659 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:49,661 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:49,666 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-03 02:53:49,667 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-04-03 02:53:49,668 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:49,668 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:49,668 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:49,668 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:49,668 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:49,752 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205edc77e830b194a5e902e39b658fc9340 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205edc77e830b194a5e902e39b658fc9340 2023-04-03 02:53:49,788 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/37f0a46fe83346cc9f6e74be36561bac 2023-04-03 02:53:49,799 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/37f0a46fe83346cc9f6e74be36561bac as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac 2023-04-03 02:53:49,814 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac, entries=1, sequenceid=17, filesize=5.0 K 2023-04-03 02:53:49,816 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 148ms, sequenceid=17, compaction requested=true 2023-04-03 02:53:49,816 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:49,816 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:49,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:49,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:49,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:49,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:49,821 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,821 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:49,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:49,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@102146d0[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:49,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:49,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,829 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,829 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:49,829 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:49,829 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:49,830 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,830 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:49,850 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:49,850 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,850 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:49,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,851 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:49,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:49,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:49,857 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:49,857 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:49,857 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:49,857 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,857 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:49,859 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,859 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:49,859 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:49,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:49,861 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:49,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,864 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:49,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,871 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:49,871 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:49,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:49,871 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:49,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:49,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:49,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:49,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:49,871 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,873 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,874 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:49,874 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:49,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:49,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:49,974 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:49,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:49,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:49,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:49,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,993 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:49,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:49,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:49,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:49,994 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,994 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,996 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,996 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:49,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:49,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:49,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:49,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:49,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,997 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:49,999 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:50,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,000 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:50,000 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:50,001 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:50,004 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:50,004 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:50,004 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:50,004 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:50,004 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:50,057 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284773a385106c4f919e0fa068da66831d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284773a385106c4f919e0fa068da66831d 2023-04-03 02:53:50,104 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/c21775a2288f4c11947651c7739e8047 2023-04-03 02:53:50,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/c21775a2288f4c11947651c7739e8047 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047 2023-04-03 02:53:50,129 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047, entries=1, sequenceid=21, filesize=5.0 K 2023-04-03 02:53:50,130 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 126ms, sequenceid=21, compaction requested=true 2023-04-03 02:53:50,130 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:50,130 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:50,130 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:50,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,137 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,137 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:50,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:50,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c2ba8e0[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:50,139 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:50,139 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,140 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,140 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:50,140 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:50,140 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:50,141 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,141 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:50,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:50,143 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:50,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:50,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:50,151 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:50,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:50,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:50,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,151 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:50,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:50,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:50,154 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,155 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,156 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:50,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:50,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,160 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:50,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:50,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,160 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,161 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:50,161 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:50,261 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:50,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:50,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:50,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:50,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,282 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:50,282 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:50,283 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:50,283 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:50,283 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,283 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,285 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,285 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:50,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:50,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:50,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:50,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:50,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,287 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:50,287 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:50,288 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:50,305 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:50,305 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:50,305 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:50,305 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:50,305 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:50,350 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231299cf6cde9b640b59923aa8fad47006c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231299cf6cde9b640b59923aa8fad47006c 2023-04-03 02:53:50,774 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/1a33980ba4aa4e02beaa24c44f6d0109 2023-04-03 02:53:50,788 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/1a33980ba4aa4e02beaa24c44f6d0109 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109 2023-04-03 02:53:50,800 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109, entries=1, sequenceid=25, filesize=5.0 K 2023-04-03 02:53:50,802 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 497ms, sequenceid=25, compaction requested=true 2023-04-03 02:53:50,802 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:50,802 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:50,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:50,803 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:50,803 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,803 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:50,803 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:50,806 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,806 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,806 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,807 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:50,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:50,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@180a95ad[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:50,809 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:50,809 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,810 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:50,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:50,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:50,811 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,811 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:50,812 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:50,812 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,812 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:50,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,812 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:50,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:50,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:50,820 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:50,820 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:50,820 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:50,820 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,820 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:50,822 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,822 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:50,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:50,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,822 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:50,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:50,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:50,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:50,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,823 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:50,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:50,826 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,831 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:50,831 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,832 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:50,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:50,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:50,832 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,832 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:50,833 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:50,833 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:50,834 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:50,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:50,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:50,934 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:50,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:50,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:50,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:50,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,952 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:50,952 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:50,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:50,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:50,955 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,955 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,958 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:50,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:50,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:50,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,964 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,965 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:50,965 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:50,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:50,972 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:50,972 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:50,972 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:50,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:50,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:50,973 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:50,974 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:50,974 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:50,974 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:51,468 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101cdd6c0b1bedb4467aa897212ae58db44 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101cdd6c0b1bedb4467aa897212ae58db44 2023-04-03 02:53:51,528 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/826c6cde12864b8589e0708e561859da 2023-04-03 02:53:51,541 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/826c6cde12864b8589e0708e561859da as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da 2023-04-03 02:53:51,560 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da, entries=1, sequenceid=29, filesize=5.0 K 2023-04-03 02:53:51,562 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 588ms, sequenceid=29, compaction requested=true 2023-04-03 02:53:51,562 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:51,563 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:51,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:51,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:51,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:51,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:51,566 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,566 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,567 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,567 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:51,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:51,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@693f4cc2[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:51,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:51,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,576 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:51,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:51,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:51,577 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,578 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:51,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:51,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:51,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:51,580 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:51,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:51,585 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:51,585 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:51,585 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:51,585 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,585 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:51,588 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,588 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,588 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:51,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,590 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,592 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:51,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:51,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:51,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,599 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:51,599 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:51,600 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,600 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:51,600 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,601 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:51,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:51,701 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:51,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:51,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:51,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:51,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,725 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:51,725 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:51,725 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:51,725 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:51,728 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,728 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,730 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:51,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:51,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:51,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:51,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,736 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:51,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,737 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:51,738 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:51,740 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:51,741 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:51,741 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:51,741 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:51,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:51,866 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010392b6ba497028421ba66fe7b850ec0888 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010392b6ba497028421ba66fe7b850ec0888 2023-04-03 02:53:51,894 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/26b6791a63534000958174bc0d3eabcf 2023-04-03 02:53:51,905 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/26b6791a63534000958174bc0d3eabcf as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf 2023-04-03 02:53:51,915 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf, entries=1, sequenceid=33, filesize=5.0 K 2023-04-03 02:53:51,918 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 176ms, sequenceid=33, compaction requested=true 2023-04-03 02:53:51,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:51,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:51,918 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:51,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:51,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:51,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:51,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:51,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:51,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73b290f3[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:51,927 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:51,927 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,930 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,930 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:51,930 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:51,930 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:51,930 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,930 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:51,931 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:51,931 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,931 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:51,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,931 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:51,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:51,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:51,936 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:51,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:51,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:51,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,936 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:51,939 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,939 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,939 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:51,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:51,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:51,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:51,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:51,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:51,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:51,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,945 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,949 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:51,949 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:51,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:51,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:51,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:51,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,952 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:51,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:51,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,952 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:51,953 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:51,954 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:52,054 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:52,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:52,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:52,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:52,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,070 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:52,070 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:52,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:52,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:52,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,075 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:52,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:52,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:52,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:52,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:52,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:52,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,078 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:52,078 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:52,079 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:52,079 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:52,080 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:52,528 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907efa52ddfe7df43d0add4cf48fee68134 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907efa52ddfe7df43d0add4cf48fee68134 2023-04-03 02:53:52,947 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/f8f0622bfa684828ac0d9b10566055d0 2023-04-03 02:53:52,956 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/f8f0622bfa684828ac0d9b10566055d0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0 2023-04-03 02:53:52,968 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0, entries=1, sequenceid=37, filesize=5.0 K 2023-04-03 02:53:52,970 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 890ms, sequenceid=37, compaction requested=true 2023-04-03 02:53:52,970 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:52,970 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:52,970 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:52,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:52,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:52,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:52,973 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:52,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:52,974 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,974 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:52,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:52,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:52,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:52,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52ff4acd[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:52,976 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:52,976 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,983 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:52,983 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:52,983 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:52,984 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,984 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:52,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:52,985 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:52,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:52,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:52,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:52,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:52,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:52,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:52,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:52,989 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:52,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:52,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:52,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,989 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:52,991 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,992 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:52,992 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:52,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:52,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:52,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:52,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:52,994 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,996 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:52,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:52,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:53,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:53,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:53,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:53,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:53,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:53,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,001 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,001 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,001 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,001 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:53,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:53,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:53,001 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,001 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:53,004 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:53,004 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:53,104 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:53,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:53,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:53,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:53,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,118 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:53,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:53,119 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-03 02:53:53,119 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-03 02:53:53,120 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,120 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,122 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:53,122 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:53,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:53,122 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,122 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:53,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:53,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-03 02:53:53,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,125 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-03 02:53:53,125 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:53,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-03 02:53:53,127 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:53,127 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:53,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:53,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. started... 2023-04-03 02:53:53,128 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6b7f8c28a8619bf76181b65e5ae70d83 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:53,551 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ef4ff8540c054e66aa6aad27520bcb71 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ef4ff8540c054e66aa6aad27520bcb71 2023-04-03 02:53:53,567 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/7d6694cd0f6d4301b17c393da36de862 2023-04-03 02:53:53,576 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/7d6694cd0f6d4301b17c393da36de862 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862 2023-04-03 02:53:53,583 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862, entries=1, sequenceid=41, filesize=5.0 K 2023-04-03 02:53:53,585 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6b7f8c28a8619bf76181b65e5ae70d83 in 457ms, sequenceid=41, compaction requested=true 2023-04-03 02:53:53,585 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:53,585 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:53,585 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:53,585 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:53,585 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,586 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-03 02:53:53,586 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-03 02:53:53,587 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,587 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:53,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-03 02:53:53,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e140ac2[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:53,590 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-03 02:53:53,590 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,594 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-03 02:53:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-03 02:53:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:53,595 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,595 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-03 02:53:53,598 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-03 02:53:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:53,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-03 02:53:53,605 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-03 02:53:53,605 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:53,605 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:53,605 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,605 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:53,607 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,607 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,607 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:53,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,608 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,610 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:53,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:53,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-03 02:53:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:53,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:53,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,615 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:53,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:53,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-03 02:53:53,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:53,616 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:53,616 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:53,716 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:53,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:53,724 DEBUG [Listener at localhost/37595] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-03 02:53:53,727 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39288, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-03 02:53:53,729 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:53,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:53,744 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:53,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:53:53,745 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491033745 2023-04-03 02:53:53,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:53:53,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-04-03 02:53:53,755 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:53,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x4a57b8a1 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:53,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c0608db, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:53,772 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,300 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010343cd831e19384007b5aebdeb89995085 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010343cd831e19384007b5aebdeb89995085 2023-04-03 02:53:54,302 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:54,333 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:53:54,335 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47932, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:53:54,349 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:53:54,351 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39292, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:53:54,393 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4734d80bb84141d8a526d3dae5b2dcf3 first=Optional[r3] last=Optional[r6] 2023-04-03 02:53:54,407 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4734d80bb84141d8a526d3dae5b2dcf3}] 2023-04-03 02:53:54,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4734d80bb84141d8a526d3dae5b2dcf3 for inclusion in 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-03 02:53:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:53:54,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:54,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4734d80bb84141d8a526d3dae5b2dcf3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fvporvuqdcd1kinq62f2c3db3i2dts3cisrt86cd8k3lfu0v2g8econnue1f32t8/family1/4734d80bb84141d8a526d3dae5b2dcf3 2023-04-03 02:53:54,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fvporvuqdcd1kinq62f2c3db3i2dts3cisrt86cd8k3lfu0v2g8econnue1f32t8/family1/4734d80bb84141d8a526d3dae5b2dcf3 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_ 2023-04-03 02:53:54,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fvporvuqdcd1kinq62f2c3db3i2dts3cisrt86cd8k3lfu0v2g8econnue1f32t8/family1/4734d80bb84141d8a526d3dae5b2dcf3 into 6b7f8c28a8619bf76181b65e5ae70d83/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_ - updating store file list. 2023-04-03 02:53:54,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_ into 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:54,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fvporvuqdcd1kinq62f2c3db3i2dts3cisrt86cd8k3lfu0v2g8econnue1f32t8/family1/4734d80bb84141d8a526d3dae5b2dcf3 into 6b7f8c28a8619bf76181b65e5ae70d83/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_) 2023-04-03 02:53:54,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__fvporvuqdcd1kinq62f2c3db3i2dts3cisrt86cd8k3lfu0v2g8econnue1f32t8/family1/4734d80bb84141d8a526d3dae5b2dcf3 2023-04-03 02:53:54,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:53:54,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:53:54,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 6b7f8c28a8619bf76181b65e5ae70d83/family1 is initiating minor compaction 2023-04-03 02:53:54,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@9b86d34; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:53:54,474 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6b7f8c28a8619bf76181b65e5ae70d83/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:54,475 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp, totalSize=50.2 K 2023-04-03 02:53:54,476 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7bcd93380365490fb328928d35c5161c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:53:54,477 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 95953b1daf934cef8795705b13797865, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 37f0a46fe83346cc9f6e74be36561bac, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting c21775a2288f4c11947651c7739e8047, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 1a33980ba4aa4e02beaa24c44f6d0109, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 826c6cde12864b8589e0708e561859da, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 26b6791a63534000958174bc0d3eabcf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f8f0622bfa684828ac0d9b10566055d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:53:54,482 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7d6694cd0f6d4301b17c393da36de862, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:53:54,483 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting b76f648858c0447480320f862b345064_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:53:54,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:54,504 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:53:54,511 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284773a385106c4f919e0fa068da66831d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512284773a385106c4f919e0fa068da66831d 2023-04-03 02:53:54,514 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231299cf6cde9b640b59923aa8fad47006c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231299cf6cde9b640b59923aa8fad47006c 2023-04-03 02:53:54,518 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101cdd6c0b1bedb4467aa897212ae58db44 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101cdd6c0b1bedb4467aa897212ae58db44 2023-04-03 02:53:54,520 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010392b6ba497028421ba66fe7b850ec0888 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010392b6ba497028421ba66fe7b850ec0888 2023-04-03 02:53:54,520 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-04-03 02:53:54,520 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,521 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:54,584 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b7f8c28a8619bf76181b65e5ae70d83#family1#compaction#10 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:53:55,059 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/60104c7cb4c945fa8492afeefacfc467 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/60104c7cb4c945fa8492afeefacfc467 2023-04-03 02:53:55,078 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 6b7f8c28a8619bf76181b65e5ae70d83/family1 of 6b7f8c28a8619bf76181b65e5ae70d83 into 60104c7cb4c945fa8492afeefacfc467(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:53:55,078 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:55,079 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1680490434469; duration=0sec 2023-04-03 02:53:55,081 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:53:55,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_] to archive 2023-04-03 02:53:55,161 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:53:55,164 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7bcd93380365490fb328928d35c5161c 2023-04-03 02:53:55,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/95953b1daf934cef8795705b13797865 2023-04-03 02:53:55,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/37f0a46fe83346cc9f6e74be36561bac 2023-04-03 02:53:55,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/c21775a2288f4c11947651c7739e8047 2023-04-03 02:53:55,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/1a33980ba4aa4e02beaa24c44f6d0109 2023-04-03 02:53:55,181 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/826c6cde12864b8589e0708e561859da 2023-04-03 02:53:55,183 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/26b6791a63534000958174bc0d3eabcf 2023-04-03 02:53:55,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/f8f0622bfa684828ac0d9b10566055d0 2023-04-03 02:53:55,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7d6694cd0f6d4301b17c393da36de862 2023-04-03 02:53:55,192 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/b76f648858c0447480320f862b345064_SeqId_44_ 2023-04-03 02:53:55,511 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055b663259d4394c01897a4a966732992c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055b663259d4394c01897a4a966732992c 2023-04-03 02:53:55,515 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:55,622 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd3d08cf3b4b47658444f7705414a273 first=Optional[r00] last=Optional[r2] 2023-04-03 02:53:55,624 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd3d08cf3b4b47658444f7705414a273}] 2023-04-03 02:53:55,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd3d08cf3b4b47658444f7705414a273 for inclusion in 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:55,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-03 02:53:55,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:53:55,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:55,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bd3d08cf3b4b47658444f7705414a273 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__20ljq1e9srj0dv214slo1ke7v450m54d7s67npbu8s3hkmmekr2n6p376dr9usoh/family1/bd3d08cf3b4b47658444f7705414a273 2023-04-03 02:53:55,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__20ljq1e9srj0dv214slo1ke7v450m54d7s67npbu8s3hkmmekr2n6p376dr9usoh/family1/bd3d08cf3b4b47658444f7705414a273 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_ 2023-04-03 02:53:55,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__20ljq1e9srj0dv214slo1ke7v450m54d7s67npbu8s3hkmmekr2n6p376dr9usoh/family1/bd3d08cf3b4b47658444f7705414a273 into 6b7f8c28a8619bf76181b65e5ae70d83/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_ - updating store file list. 2023-04-03 02:53:55,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_ into 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:55,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__20ljq1e9srj0dv214slo1ke7v450m54d7s67npbu8s3hkmmekr2n6p376dr9usoh/family1/bd3d08cf3b4b47658444f7705414a273 into 6b7f8c28a8619bf76181b65e5ae70d83/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_) 2023-04-03 02:53:55,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__20ljq1e9srj0dv214slo1ke7v450m54d7s67npbu8s3hkmmekr2n6p376dr9usoh/family1/bd3d08cf3b4b47658444f7705414a273 2023-04-03 02:53:55,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:53:55,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:53:55,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 6b7f8c28a8619bf76181b65e5ae70d83/family1 is initiating minor compaction (all files) 2023-04-03 02:53:55,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5f3689d6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:53:55,654 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6b7f8c28a8619bf76181b65e5ae70d83/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:55,655 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/60104c7cb4c945fa8492afeefacfc467, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp, totalSize=358.6 K 2023-04-03 02:53:55,656 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 01b0499e3b784edd8803b49b0b15f17c, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:53:55,656 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 60104c7cb4c945fa8492afeefacfc467, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:53:55,657 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878844911 2023-04-03 02:53:55,657 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:55,659 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:53:55,661 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511306062b283020a452d87ec117b59257f91 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511306062b283020a452d87ec117b59257f91 2023-04-03 02:53:55,663 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512017cc61831b2954aaea93aa709cd76b21d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512017cc61831b2954aaea93aa709cd76b21d 2023-04-03 02:53:55,665 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205edc77e830b194a5e902e39b658fc9340 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205edc77e830b194a5e902e39b658fc9340 2023-04-03 02:53:55,665 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-04-03 02:53:55,665 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-03 02:53:55,666 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:55,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:55,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:53:55,668 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:55,671 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.9330 sec 2023-04-03 02:53:55,701 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b7f8c28a8619bf76181b65e5ae70d83#family1#compaction#11 average throughput is 5.90 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:53:55,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-03 02:53:55,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:56,384 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-04-03 02:53:56,386 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:56,386 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-04-03 02:53:56,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-04-03 02:53:56,409 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, REOPEN/MOVE}] 2023-04-03 02:53:56,410 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, REOPEN/MOVE 2023-04-03 02:53:56,411 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:56,412 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490436411"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490436411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490436411"}]},"ts":"1680490436411"} 2023-04-03 02:53:56,414 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:56,573 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6b7f8c28a8619bf76181b65e5ae70d83, disabling compactions & flushes 2023-04-03 02:53:56,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,613 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/.tmp/family1/7cb8a01c97a74cb8962ea994884ede36 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7cb8a01c97a74cb8962ea994884ede36 2023-04-03 02:53:56,621 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 6b7f8c28a8619bf76181b65e5ae70d83/family1 of 6b7f8c28a8619bf76181b65e5ae70d83 into 7cb8a01c97a74cb8962ea994884ede36(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:53:56,621 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:56,621 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. after waiting 0 ms 2023-04-03 02:53:56,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,622 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1680490435654; duration=0sec 2023-04-03 02:53:56,622 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:53:56,627 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/60104c7cb4c945fa8492afeefacfc467, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_] to archive 2023-04-03 02:53:56,628 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:53:56,630 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/01b0499e3b784edd8803b49b0b15f17c 2023-04-03 02:53:56,634 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/60104c7cb4c945fa8492afeefacfc467 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/60104c7cb4c945fa8492afeefacfc467 2023-04-03 02:53:56,636 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/0e6fdd824d0c46089a7e51c73eeea5ce_SeqId_47_ 2023-04-03 02:53:56,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-03 02:53:56,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:56,650 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 6b7f8c28a8619bf76181b65e5ae70d83 to self. 2023-04-03 02:53:56,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,655 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=CLOSED 2023-04-03 02:53:56,656 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490436655"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490436655"}]},"ts":"1680490436655"} 2023-04-03 02:53:56,661 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-04-03 02:53:56,661 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719 in 244 msec 2023-04-03 02:53:56,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=true 2023-04-03 02:53:56,813 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:56,813 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490436813"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490436813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490436813"}]},"ts":"1680490436813"} 2023-04-03 02:53:56,816 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:56,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b7f8c28a8619bf76181b65e5ae70d83, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,976 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,977 DEBUG [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:56,978 DEBUG [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:56,978 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b7f8c28a8619bf76181b65e5ae70d83 columnFamilyName family1 2023-04-03 02:53:56,987 DEBUG [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/7cb8a01c97a74cb8962ea994884ede36 2023-04-03 02:53:56,988 INFO [StoreOpener-6b7f8c28a8619bf76181b65e5ae70d83-1] regionserver.HStore(310): Store=6b7f8c28a8619bf76181b65e5ae70d83/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:56,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:53:56,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6b7f8c28a8619bf76181b65e5ae70d83; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66336389, jitterRate=-0.011510774493217468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:53:56,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:56,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., pid=17, masterSystemTime=1680490436969 2023-04-03 02:53:57,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:57,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:53:57,001 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6b7f8c28a8619bf76181b65e5ae70d83, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:57,002 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1680490437001"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490437001"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490437001"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490437001"}]},"ts":"1680490437001"} 2023-04-03 02:53:57,007 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-04-03 02:53:57,008 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 6b7f8c28a8619bf76181b65e5ae70d83, server=jenkins-hbase4.apache.org,45927,1680490420719 in 188 msec 2023-04-03 02:53:57,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-04-03 02:53:57,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=6b7f8c28a8619bf76181b65e5ae70d83, REOPEN/MOVE in 599 msec 2023-04-03 02:53:57,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-04-03 02:53:57,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 621 msec 2023-04-03 02:53:57,017 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0690 sec 2023-04-03 02:53:57,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-04-03 02:53:57,028 INFO [Listener at localhost/37595] compactions.TestMobCompactor(278): alter status finished 2023-04-03 02:53:57,029 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-03 02:53:57,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:57,031 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:57,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:53:57,032 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491037032 2023-04-03 02:53:57,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:53:57,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-04-03 02:53:57,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:57,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x0b052bf3 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:53:57,060 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10316ea4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:53:57,061 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,061 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,061 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,061 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,061 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,129 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286de2c3e883da45368371bc9beeac277d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511286de2c3e883da45368371bc9beeac277d 2023-04-03 02:53:57,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:57,150 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:53:57,152 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47938, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:53:57,166 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:53:57,169 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39308, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:53:57,186 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/455b51ab07a4490aa754e445f77ec1ab first=Optional[r05] last=Optional[r8] 2023-04-03 02:53:57,188 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=51 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/455b51ab07a4490aa754e445f77ec1ab}] 2023-04-03 02:53:57,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/455b51ab07a4490aa754e445f77ec1ab for inclusion in 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:57,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-03 02:53:57,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:53:57,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:53:57,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/455b51ab07a4490aa754e445f77ec1ab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__d4akdf65b5ncer4iokgdhp0k6t1bka0sbe3pmemh92kc23kf896jrcfm6afn25a0/family1/455b51ab07a4490aa754e445f77ec1ab 2023-04-03 02:53:57,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__d4akdf65b5ncer4iokgdhp0k6t1bka0sbe3pmemh92kc23kf896jrcfm6afn25a0/family1/455b51ab07a4490aa754e445f77ec1ab as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/e3326eba0eae4fed92cd16e2da13a524_SeqId_53_ 2023-04-03 02:53:57,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__d4akdf65b5ncer4iokgdhp0k6t1bka0sbe3pmemh92kc23kf896jrcfm6afn25a0/family1/455b51ab07a4490aa754e445f77ec1ab into 6b7f8c28a8619bf76181b65e5ae70d83/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/e3326eba0eae4fed92cd16e2da13a524_SeqId_53_ - updating store file list. 2023-04-03 02:53:57,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/e3326eba0eae4fed92cd16e2da13a524_SeqId_53_ into 6b7f8c28a8619bf76181b65e5ae70d83/family1 2023-04-03 02:53:57,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__d4akdf65b5ncer4iokgdhp0k6t1bka0sbe3pmemh92kc23kf896jrcfm6afn25a0/family1/455b51ab07a4490aa754e445f77ec1ab into 6b7f8c28a8619bf76181b65e5ae70d83/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/family1/e3326eba0eae4fed92cd16e2da13a524_SeqId_53_) 2023-04-03 02:53:57,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__d4akdf65b5ncer4iokgdhp0k6t1bka0sbe3pmemh92kc23kf896jrcfm6afn25a0/family1/455b51ab07a4490aa754e445f77ec1ab 2023-04-03 02:53:57,221 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:57,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:53:57,224 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ef4ff8540c054e66aa6aad27520bcb71 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ef4ff8540c054e66aa6aad27520bcb71 2023-04-03 02:53:57,225 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128608129eb036f463a9e4a3a9bbfd584b0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128608129eb036f463a9e4a3a9bbfd584b0 2023-04-03 02:53:57,225 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-04-03 02:53:57,225 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-03 02:53:57,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,226 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-03 02:53:57,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-03 02:53:57,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:53:57,228 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-03 02:53:57,231 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 198 msec 2023-04-03 02:53:57,368 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=251 (was 210) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:48390 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1aaa96c0-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35780 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4597457-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35712 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35762 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35810 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4597457-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x0b052bf3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/523480629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35690 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1aaa96c0-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35674 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35856 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x4a57b8a1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35682 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35658 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x0b052bf3-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x4597457-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35854 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4597457-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35804 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x4a57b8a1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/523480629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1aaa96c0-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:48426 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35772 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x0b052bf3-SendThread(127.0.0.1:52880) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS:0;jenkins-hbase4:45927-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272019329_15 at /127.0.0.1:35672 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x4a57b8a1-SendThread(127.0.0.1:52880) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35696 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4597457-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=596 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=514 (was 442) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=3871 (was 4328) 2023-04-03 02:53:57,397 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=514, ProcessCount=170, AvailableMemoryMB=3870 2023-04-03 02:53:57,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-03 02:53:57,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:57,405 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:53:57,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-04-03 02:53:57,407 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:53:57,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-03 02:53:57,428 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,429 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a empty. 2023-04-03 02:53:57,432 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,432 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-04-03 02:53:57,474 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-03 02:53:57,478 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5a1b3698dfeedaa8cd615262240ef68a, NAME => 'testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:53:57,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-03 02:53:57,535 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:57,535 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 5a1b3698dfeedaa8cd615262240ef68a, disabling compactions & flushes 2023-04-03 02:53:57,536 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,536 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,536 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. after waiting 0 ms 2023-04-03 02:53:57,536 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,536 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,536 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:53:57,540 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:53:57,541 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490437541"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490437541"}]},"ts":"1680490437541"} 2023-04-03 02:53:57,546 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:53:57,552 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:53:57,552 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490437552"}]},"ts":"1680490437552"} 2023-04-03 02:53:57,554 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-04-03 02:53:57,557 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a1b3698dfeedaa8cd615262240ef68a, ASSIGN}] 2023-04-03 02:53:57,560 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a1b3698dfeedaa8cd615262240ef68a, ASSIGN 2023-04-03 02:53:57,562 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a1b3698dfeedaa8cd615262240ef68a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:53:57,713 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5a1b3698dfeedaa8cd615262240ef68a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:57,714 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490437713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490437713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490437713"}]},"ts":"1680490437713"} 2023-04-03 02:53:57,716 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 5a1b3698dfeedaa8cd615262240ef68a, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:53:57,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-03 02:53:57,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5a1b3698dfeedaa8cd615262240ef68a, NAME => 'testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:53:57,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:53:57,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,876 INFO [StoreOpener-5a1b3698dfeedaa8cd615262240ef68a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,878 DEBUG [StoreOpener-5a1b3698dfeedaa8cd615262240ef68a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:53:57,878 DEBUG [StoreOpener-5a1b3698dfeedaa8cd615262240ef68a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:53:57,879 INFO [StoreOpener-5a1b3698dfeedaa8cd615262240ef68a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a1b3698dfeedaa8cd615262240ef68a columnFamilyName family1 2023-04-03 02:53:57,880 INFO [StoreOpener-5a1b3698dfeedaa8cd615262240ef68a-1] regionserver.HStore(310): Store=5a1b3698dfeedaa8cd615262240ef68a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:53:57,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:53:57,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:53:57,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5a1b3698dfeedaa8cd615262240ef68a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74542448, jitterRate=0.11076903343200684}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:53:57,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:53:57,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., pid=21, masterSystemTime=1680490437869 2023-04-03 02:53:57,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:57,892 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5a1b3698dfeedaa8cd615262240ef68a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:57,892 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490437892"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490437892"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490437892"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490437892"}]},"ts":"1680490437892"} 2023-04-03 02:53:57,897 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-04-03 02:53:57,897 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 5a1b3698dfeedaa8cd615262240ef68a, server=jenkins-hbase4.apache.org,45927,1680490420719 in 178 msec 2023-04-03 02:53:57,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-04-03 02:53:57,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a1b3698dfeedaa8cd615262240ef68a, ASSIGN in 340 msec 2023-04-03 02:53:57,900 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:53:57,900 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490437900"}]},"ts":"1680490437900"} 2023-04-03 02:53:57,902 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-04-03 02:53:57,906 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:53:57,908 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 506 msec 2023-04-03 02:53:58,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-03 02:53:58,022 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-04-03 02:53:58,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:58,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,068 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,068 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:53:58,069 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,070 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,071 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:58,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:58,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:58,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:58,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,073 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,077 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:53:58,077 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:58,077 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:58,077 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:58,077 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:53:58,078 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:53:58,122 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c93056671e8d43b89c11d8dbe1801e5d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c93056671e8d43b89c11d8dbe1801e5d 2023-04-03 02:53:58,127 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/e8b363c4e2d046ddb778cb9cc7a28c16 2023-04-03 02:53:58,135 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/e8b363c4e2d046ddb778cb9cc7a28c16 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16 2023-04-03 02:53:58,141 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16, entries=1000, sequenceid=5, filesize=166.0 K 2023-04-03 02:53:58,143 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 65ms, sequenceid=5, compaction requested=false 2023-04-03 02:53:58,143 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,144 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:53:58,144 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:53:58,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,146 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,147 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,147 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dde1c9e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:58,149 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:53:58,149 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,150 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,152 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:53:58,152 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:53:58,152 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:58,156 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,156 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:58,158 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:53:58,158 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,158 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:58,158 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:53:58,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:58,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:53:58,164 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:53:58,164 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:58,164 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:58,164 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,164 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:58,166 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,166 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:58,166 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:58,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,168 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,170 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,173 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,174 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:58,174 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:58,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:58,174 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:58,174 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:58,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:58,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:58,275 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:58,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:58,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:58,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:58,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,290 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,290 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,291 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:53:58,291 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:53:58,291 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,291 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,294 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,294 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:58,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:58,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:58,295 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,295 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:58,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:58,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,312 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:53:58,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,312 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:53:58,312 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,313 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:53:58,318 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:58,319 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:58,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:58,319 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:53:58,319 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:53:58,350 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b660269567544cf7848bbf2c6533193b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b660269567544cf7848bbf2c6533193b 2023-04-03 02:53:58,763 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/a125515b97cd413ab29cff145e49dc8f 2023-04-03 02:53:58,778 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/a125515b97cd413ab29cff145e49dc8f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f 2023-04-03 02:53:58,787 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:53:58,801 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 470ms, sequenceid=9, compaction requested=false 2023-04-03 02:53:58,802 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:53:58,802 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:53:58,802 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:53:58,804 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,804 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,804 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,805 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a8c20de[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:58,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:53:58,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,809 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:58,810 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,810 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:58,811 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:58,811 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:53:58,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:53:58,816 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:53:58,816 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:58,816 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:58,816 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,816 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:58,821 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,821 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:58,821 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:58,822 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,824 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,835 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,835 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,835 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:58,835 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:58,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:58,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:53:58,835 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:58,835 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:58,835 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,836 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,836 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,836 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,836 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,836 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:58,837 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:58,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:58,937 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:58,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:58,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:58,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:58,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,950 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:58,950 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,951 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:53:58,951 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:53:58,951 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,951 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,953 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:58,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:58,953 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:58,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:58,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:58,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,955 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,956 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:53:58,957 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:58,957 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:53:58,957 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:58,957 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:53:58,958 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:58,958 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:58,958 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:58,958 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:53:58,958 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:53:58,985 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e01e75291d474ad0aa09a155e8f2dbef to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e01e75291d474ad0aa09a155e8f2dbef 2023-04-03 02:53:59,393 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/80d0e8c996b246b4b743e70d0473c9db 2023-04-03 02:53:59,399 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/80d0e8c996b246b4b743e70d0473c9db as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db 2023-04-03 02:53:59,405 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db, entries=1, sequenceid=13, filesize=4.9 K 2023-04-03 02:53:59,407 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 449ms, sequenceid=13, compaction requested=true 2023-04-03 02:53:59,407 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:53:59,407 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:59,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:53:59,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:53:59,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:53:59,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:53:59,457 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,457 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:59,457 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,457 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:53:59,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:59,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:53:59,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@420da660[Count = 0] remaining members to acquire global barrier 2023-04-03 02:53:59,459 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:53:59,459 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,504 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,504 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,504 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:53:59,504 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:53:59,504 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:53:59,504 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:53:59,564 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:53:59,565 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:59,564 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,565 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:53:59,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:59,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:53:59,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:53:59,570 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:53:59,570 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:53:59,570 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:53:59,570 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,570 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:53:59,611 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,611 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,611 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:53:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:53:59,611 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:53:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:53:59,613 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:53:59,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:59,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:59,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:53:59,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:59,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:53:59,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:59,660 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:53:59,660 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:53:59,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:53:59,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:53:59,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:53:59,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:53:59,760 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:53:59,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:53:59,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:53:59,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:53:59,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,770 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:53:59,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:59,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:53:59,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:53:59,771 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,771 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,819 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:53:59,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:53:59,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:53:59,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:53:59,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:53:59,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:53:59,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,821 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,821 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:53:59,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:53:59,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:53:59,823 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:53:59,823 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:53:59,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:53:59,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:53:59,823 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:00,242 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205be7c4fedcd424f689413f2b98b5f8b4e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205be7c4fedcd424f689413f2b98b5f8b4e 2023-04-03 02:54:00,657 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/2feffe432db9466d9af0cdbb81744431 2023-04-03 02:54:00,665 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/2feffe432db9466d9af0cdbb81744431 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431 2023-04-03 02:54:00,671 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431, entries=1, sequenceid=17, filesize=4.9 K 2023-04-03 02:54:00,672 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 849ms, sequenceid=17, compaction requested=true 2023-04-03 02:54:00,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:00,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:00,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:00,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:00,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:00,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:00,675 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:00,675 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:00,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:00,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c71f6f9[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:00,677 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:00,677 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,678 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,678 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:00,678 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:00,678 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:00,679 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,679 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:00,680 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:00,680 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,680 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:00,680 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:00,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:00,684 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:00,684 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:00,684 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:00,684 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,684 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:00,686 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,686 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,686 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:00,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:00,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:00,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:00,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,691 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,694 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:00,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:00,697 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:00,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:00,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,698 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:00,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:00,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:00,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:00,698 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:00,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:00,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:00,799 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:00,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:00,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:00,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:00,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,807 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:00,807 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:00,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:00,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:00,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,808 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,809 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,809 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:00,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:00,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:00,810 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:00,810 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:00,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,818 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:00,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:00,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:00,828 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:00,828 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:00,829 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:00,844 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:00,844 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:00,848 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:00,849 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:00,849 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:01,281 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280dc309ab337d4e04bda77155c5b21965 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512280dc309ab337d4e04bda77155c5b21965 2023-04-03 02:54:01,288 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/7a9ee274d01840929abc47c89fc90dc5 2023-04-03 02:54:01,298 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/7a9ee274d01840929abc47c89fc90dc5 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5 2023-04-03 02:54:01,305 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5, entries=1, sequenceid=21, filesize=4.9 K 2023-04-03 02:54:01,306 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 457ms, sequenceid=21, compaction requested=true 2023-04-03 02:54:01,307 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:01,307 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:01,307 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:01,307 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:01,307 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,307 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:01,307 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:01,309 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,309 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,310 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:01,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b62f05[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:01,311 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:01,311 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,313 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,313 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:01,313 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:01,313 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:01,313 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,313 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:01,314 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:01,314 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,314 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:01,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,315 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:01,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:01,318 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:01,318 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:01,318 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:01,318 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,318 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:01,323 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,323 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,324 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,324 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,326 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,329 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,329 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:01,329 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:01,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,330 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:01,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:01,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:01,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:01,330 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:01,332 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:01,332 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:01,432 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:01,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:01,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:01,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:01,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,442 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:01,442 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:01,442 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:01,442 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:01,443 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,443 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,444 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,444 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:01,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:01,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:01,445 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,445 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:01,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:01,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,446 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:01,446 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:01,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:01,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:01,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:01,447 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:01,447 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:01,447 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:01,881 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310f0c83ebb0a54ca08fc04f0ed16e3424 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512310f0c83ebb0a54ca08fc04f0ed16e3424 2023-04-03 02:54:01,888 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/8ecf76056f0a4bfbb4de942ef7042ecf 2023-04-03 02:54:01,895 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/8ecf76056f0a4bfbb4de942ef7042ecf as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf 2023-04-03 02:54:01,902 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf, entries=1, sequenceid=25, filesize=4.9 K 2023-04-03 02:54:01,903 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 456ms, sequenceid=25, compaction requested=true 2023-04-03 02:54:01,903 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:01,903 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:01,903 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:01,903 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:01,903 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,903 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:01,903 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:01,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,906 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,906 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:01,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a37653d[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:01,907 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:01,907 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,908 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,908 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:01,908 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:01,908 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:01,908 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,909 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:01,911 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:01,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:01,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:01,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:01,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:01,914 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:01,914 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:01,914 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:01,914 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,914 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:01,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:01,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,916 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:01,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:01,918 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:01,920 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:01,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:01,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:01,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:01,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:01,921 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:01,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:54:01,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:01,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:01,921 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:02,022 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:02,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:02,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:02,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:02,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,032 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:02,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:02,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:02,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:02,033 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,033 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,034 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:02,034 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:02,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:02,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,038 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:02,038 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:02,038 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:02,039 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:02,039 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:02,039 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:02,183 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:54:02,463 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101dffa9b92710249c48512d3af879c35b4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101dffa9b92710249c48512d3af879c35b4 2023-04-03 02:54:02,870 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/abcb28e369df4243aed011f221e038b6 2023-04-03 02:54:02,879 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/abcb28e369df4243aed011f221e038b6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6 2023-04-03 02:54:02,886 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6, entries=1, sequenceid=29, filesize=4.9 K 2023-04-03 02:54:02,887 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 848ms, sequenceid=29, compaction requested=true 2023-04-03 02:54:02,887 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:02,887 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:02,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:02,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:02,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:02,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:02,889 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:02,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:02,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:02,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:02,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:02,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:02,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2526ecbd[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:02,891 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:02,891 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,893 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,893 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:02,893 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:02,893 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:02,893 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,893 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:02,895 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,895 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,895 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:02,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:02,895 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:02,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:02,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:02,899 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:02,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:02,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:02,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,899 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:02,902 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,902 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:02,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:02,902 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:02,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:02,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:02,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,904 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,905 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:02,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:02,909 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:02,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:02,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:02,910 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,910 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:02,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:02,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:02,912 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:02,912 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:03,012 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:03,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:03,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:03,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:03,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,024 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,025 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,025 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:03,025 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:03,033 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,033 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,034 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,036 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,036 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:03,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:03,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:03,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,037 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:03,037 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,037 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:03,038 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:03,038 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:03,038 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,038 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:03,038 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:03,065 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038a9ee460a5794e3a8293c93b02641b46 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038a9ee460a5794e3a8293c93b02641b46 2023-04-03 02:54:03,084 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/c2e707ced80244fe9e390b5b06edc1c6 2023-04-03 02:54:03,094 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/c2e707ced80244fe9e390b5b06edc1c6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6 2023-04-03 02:54:03,100 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6, entries=1, sequenceid=33, filesize=4.9 K 2023-04-03 02:54:03,101 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 63ms, sequenceid=33, compaction requested=true 2023-04-03 02:54:03,102 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:03,102 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,102 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:03,102 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:03,102 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,102 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:03,102 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:03,104 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:03,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@561300de[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:03,106 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:03,107 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,108 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,108 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:03,108 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:03,108 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:03,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:03,112 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:03,116 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:03,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:03,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:03,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,116 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:03,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,120 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,122 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,125 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:03,125 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:54:03,126 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:03,126 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:03,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,226 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:03,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:03,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:03,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:03,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,246 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,246 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,246 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:03,246 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:03,247 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,247 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,249 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,249 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,249 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,249 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,250 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:03,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,251 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:03,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:03,268 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:03,268 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,268 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:03,268 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:03,268 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:03,328 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071f1b35fe3cbd4dd496ea6f1be1e769ce to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509071f1b35fe3cbd4dd496ea6f1be1e769ce 2023-04-03 02:54:03,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/30cfd721626d489c93b75aa8ad30a3e8 2023-04-03 02:54:03,361 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/30cfd721626d489c93b75aa8ad30a3e8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8 2023-04-03 02:54:03,375 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8, entries=1, sequenceid=37, filesize=4.9 K 2023-04-03 02:54:03,378 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 110ms, sequenceid=37, compaction requested=true 2023-04-03 02:54:03,378 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:03,378 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:03,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:03,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:03,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:03,382 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,382 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,383 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,383 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:03,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:03,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66edc31e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:03,388 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:03,388 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,390 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,390 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:03,390 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:03,390 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:03,391 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,391 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:03,392 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:03,392 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,392 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,392 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:03,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:03,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:03,400 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:03,400 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:03,400 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:03,400 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,400 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:03,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,403 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,406 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,409 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,410 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:03,410 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:03,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:54:03,511 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:03,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:03,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:03,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:03,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,521 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,521 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,522 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:54:03,522 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:03,522 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,522 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,524 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,524 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,525 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,525 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:03,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,525 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,526 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,526 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:54:03,526 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,526 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:54:03,526 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:03,528 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:54:03,528 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:03,528 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:03,528 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,528 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. started... 2023-04-03 02:54:03,528 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5a1b3698dfeedaa8cd615262240ef68a 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:03,549 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511206b82145f90aa4e26b266b11fd1a5c248 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511206b82145f90aa4e26b266b11fd1a5c248 2023-04-03 02:54:03,555 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/f1d21c7835fb45259350b2022506ac48 2023-04-03 02:54:03,562 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/f1d21c7835fb45259350b2022506ac48 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48 2023-04-03 02:54:03,573 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48, entries=1, sequenceid=41, filesize=4.9 K 2023-04-03 02:54:03,574 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a1b3698dfeedaa8cd615262240ef68a in 46ms, sequenceid=41, compaction requested=true 2023-04-03 02:54:03,574 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:03,574 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:03,574 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:03,574 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:03,574 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,574 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:54:03,574 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:54:03,576 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:03,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:54:03,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57c25c73[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:03,578 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:03,578 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,579 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:03,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:54:03,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,580 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:03,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:03,581 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,581 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:54:03,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,581 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,581 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:54:03,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:03,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:03,584 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:54:03,584 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:03,584 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:03,584 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,584 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:03,587 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,587 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:03,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:03,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,587 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:03,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,588 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:03,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,589 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,592 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:03,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:03,592 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:03,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:03,592 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:03,592 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:03,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-03 02:54:03,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:03,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:03,592 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,593 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,593 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:03,593 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,594 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,594 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,594 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:03,594 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:03,694 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:03,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:03,697 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-04-03 02:54:03,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:03,699 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:03,700 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491043700 2023-04-03 02:54:03,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:03,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:03,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-04-03 02:54:03,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:03,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x7447ff9a to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:03,715 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f18ebf8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:03,716 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:04,172 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039c44e16ffa3c4394b1845e9d391079b8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601039c44e16ffa3c4394b1845e9d391079b8 2023-04-03 02:54:04,173 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:04,205 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:04,207 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34242, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:04,227 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:04,228 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34592, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:04,263 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3e91b691e2a446bd95f5674f81c5a12e first=Optional[r5] last=Optional[r6] 2023-04-03 02:54:04,264 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3e91b691e2a446bd95f5674f81c5a12e}] 2023-04-03 02:54:04,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3e91b691e2a446bd95f5674f81c5a12e for inclusion in 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:04,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-04-03 02:54:04,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:04,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:04,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3e91b691e2a446bd95f5674f81c5a12e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__bgddn8757rfaqfbr669fkhb11c0mg19v0t2mh8a1u14miktlfg9ggvr17ui838f6/family1/3e91b691e2a446bd95f5674f81c5a12e 2023-04-03 02:54:04,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__bgddn8757rfaqfbr669fkhb11c0mg19v0t2mh8a1u14miktlfg9ggvr17ui838f6/family1/3e91b691e2a446bd95f5674f81c5a12e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_ 2023-04-03 02:54:04,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__bgddn8757rfaqfbr669fkhb11c0mg19v0t2mh8a1u14miktlfg9ggvr17ui838f6/family1/3e91b691e2a446bd95f5674f81c5a12e into 5a1b3698dfeedaa8cd615262240ef68a/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_ - updating store file list. 2023-04-03 02:54:04,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_ into 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:04,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__bgddn8757rfaqfbr669fkhb11c0mg19v0t2mh8a1u14miktlfg9ggvr17ui838f6/family1/3e91b691e2a446bd95f5674f81c5a12e into 5a1b3698dfeedaa8cd615262240ef68a/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_) 2023-04-03 02:54:04,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__bgddn8757rfaqfbr669fkhb11c0mg19v0t2mh8a1u14miktlfg9ggvr17ui838f6/family1/3e91b691e2a446bd95f5674f81c5a12e 2023-04-03 02:54:04,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:54:04,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:54:04,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 5a1b3698dfeedaa8cd615262240ef68a/family1 is initiating minor compaction 2023-04-03 02:54:04,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6f44e718; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:04,295 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5a1b3698dfeedaa8cd615262240ef68a/family1 in testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:04,296 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp, totalSize=49.7 K 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting a125515b97cd413ab29cff145e49dc8f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 80d0e8c996b246b4b743e70d0473c9db, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 2feffe432db9466d9af0cdbb81744431, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7a9ee274d01840929abc47c89fc90dc5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 8ecf76056f0a4bfbb4de942ef7042ecf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting abcb28e369df4243aed011f221e038b6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting c2e707ced80244fe9e390b5b06edc1c6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 30cfd721626d489c93b75aa8ad30a3e8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:54:04,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f1d21c7835fb45259350b2022506ac48, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:54:04,297 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:54:04,299 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:04,300 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:04,304 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101dffa9b92710249c48512d3af879c35b4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101dffa9b92710249c48512d3af879c35b4 2023-04-03 02:54:04,307 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038a9ee460a5794e3a8293c93b02641b46 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601038a9ee460a5794e3a8293c93b02641b46 2023-04-03 02:54:04,307 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-04-03 02:54:04,307 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-04-03 02:54:04,308 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:04,308 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:04,428 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5a1b3698dfeedaa8cd615262240ef68a#family1#compaction#22 average throughput is 0.46 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:04,834 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310aa947e144bd4393a125d6de44ec5273 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512310aa947e144bd4393a125d6de44ec5273 2023-04-03 02:54:04,835 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:04,874 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/33605e5958bf4639be83ca22afb87a5c first=Optional[r1] last=Optional[r4] 2023-04-03 02:54:04,874 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/33605e5958bf4639be83ca22afb87a5c}] 2023-04-03 02:54:04,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/33605e5958bf4639be83ca22afb87a5c for inclusion in 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:04,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-04-03 02:54:04,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:04,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:04,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/33605e5958bf4639be83ca22afb87a5c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__fkiqvrcqlanbgjskuu4hv242vgnlbpe7j7sr0h7klh2tu9blvu0plqobo21ppfh0/family1/33605e5958bf4639be83ca22afb87a5c 2023-04-03 02:54:04,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__fkiqvrcqlanbgjskuu4hv242vgnlbpe7j7sr0h7klh2tu9blvu0plqobo21ppfh0/family1/33605e5958bf4639be83ca22afb87a5c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_ 2023-04-03 02:54:04,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__fkiqvrcqlanbgjskuu4hv242vgnlbpe7j7sr0h7klh2tu9blvu0plqobo21ppfh0/family1/33605e5958bf4639be83ca22afb87a5c into 5a1b3698dfeedaa8cd615262240ef68a/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_ - updating store file list. 2023-04-03 02:54:04,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_ into 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:04,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__fkiqvrcqlanbgjskuu4hv242vgnlbpe7j7sr0h7klh2tu9blvu0plqobo21ppfh0/family1/33605e5958bf4639be83ca22afb87a5c into 5a1b3698dfeedaa8cd615262240ef68a/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_) 2023-04-03 02:54:04,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__fkiqvrcqlanbgjskuu4hv242vgnlbpe7j7sr0h7klh2tu9blvu0plqobo21ppfh0/family1/33605e5958bf4639be83ca22afb87a5c 2023-04-03 02:54:04,906 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:04,907 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:04,909 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e01e75291d474ad0aa09a155e8f2dbef to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e01e75291d474ad0aa09a155e8f2dbef 2023-04-03 02:54:04,910 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205be7c4fedcd424f689413f2b98b5f8b4e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205be7c4fedcd424f689413f2b98b5f8b4e 2023-04-03 02:54:04,911 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512280dc309ab337d4e04bda77155c5b21965 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512280dc309ab337d4e04bda77155c5b21965 2023-04-03 02:54:04,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512310f0c83ebb0a54ca08fc04f0ed16e3424 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512310f0c83ebb0a54ca08fc04f0ed16e3424 2023-04-03 02:54:04,913 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-04-03 02:54:04,913 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-04-03 02:54:04,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:04,913 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:05,256 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/a18ce57c7ebf4ebb93a5c49722037094 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a18ce57c7ebf4ebb93a5c49722037094 2023-04-03 02:54:05,268 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5a1b3698dfeedaa8cd615262240ef68a/family1 of 5a1b3698dfeedaa8cd615262240ef68a into a18ce57c7ebf4ebb93a5c49722037094(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:05,268 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:05,268 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1680490444295; duration=0sec 2023-04-03 02:54:05,269 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:05,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_] to archive 2023-04-03 02:54:05,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:05,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a125515b97cd413ab29cff145e49dc8f 2023-04-03 02:54:05,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/80d0e8c996b246b4b743e70d0473c9db 2023-04-03 02:54:05,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/2feffe432db9466d9af0cdbb81744431 2023-04-03 02:54:05,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/7a9ee274d01840929abc47c89fc90dc5 2023-04-03 02:54:05,359 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/8ecf76056f0a4bfbb4de942ef7042ecf 2023-04-03 02:54:05,360 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/abcb28e369df4243aed011f221e038b6 2023-04-03 02:54:05,362 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/c2e707ced80244fe9e390b5b06edc1c6 2023-04-03 02:54:05,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/30cfd721626d489c93b75aa8ad30a3e8 2023-04-03 02:54:05,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/f1d21c7835fb45259350b2022506ac48 2023-04-03 02:54:05,367 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/43c8bce9b4b7459c8daac8cd721ed8eb_SeqId_44_ 2023-04-03 02:54:05,387 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113028f7c4aac17343169688303a23120b79 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113028f7c4aac17343169688303a23120b79 2023-04-03 02:54:05,392 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:05,425 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/469f5403b623482ba7c4aa55226591bb first=Optional[r00] last=Optional[r8] 2023-04-03 02:54:05,426 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/469f5403b623482ba7c4aa55226591bb}] 2023-04-03 02:54:05,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/469f5403b623482ba7c4aa55226591bb for inclusion in 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:05,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-04-03 02:54:05,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:05,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:05,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/469f5403b623482ba7c4aa55226591bb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__uahnhbjbsuano3h0b2u72e6pi2oddsi2p1pblgg5k6flcltf49e6hjdp35lmlqvm/family1/469f5403b623482ba7c4aa55226591bb 2023-04-03 02:54:05,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__uahnhbjbsuano3h0b2u72e6pi2oddsi2p1pblgg5k6flcltf49e6hjdp35lmlqvm/family1/469f5403b623482ba7c4aa55226591bb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_ 2023-04-03 02:54:05,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__uahnhbjbsuano3h0b2u72e6pi2oddsi2p1pblgg5k6flcltf49e6hjdp35lmlqvm/family1/469f5403b623482ba7c4aa55226591bb into 5a1b3698dfeedaa8cd615262240ef68a/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_ - updating store file list. 2023-04-03 02:54:05,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_ into 5a1b3698dfeedaa8cd615262240ef68a/family1 2023-04-03 02:54:05,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__uahnhbjbsuano3h0b2u72e6pi2oddsi2p1pblgg5k6flcltf49e6hjdp35lmlqvm/family1/469f5403b623482ba7c4aa55226591bb into 5a1b3698dfeedaa8cd615262240ef68a/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_) 2023-04-03 02:54:05,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithMonthlyPolicy__uahnhbjbsuano3h0b2u72e6pi2oddsi2p1pblgg5k6flcltf49e6hjdp35lmlqvm/family1/469f5403b623482ba7c4aa55226591bb 2023-04-03 02:54:05,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-03 02:54:05,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-03 02:54:05,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 5a1b3698dfeedaa8cd615262240ef68a/family1 is initiating minor compaction (all files) 2023-04-03 02:54:05,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@282c78a5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:05,450 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5a1b3698dfeedaa8cd615262240ef68a/family1 in testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:54:05,451 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a18ce57c7ebf4ebb93a5c49722037094, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp, totalSize=344.3 K 2023-04-03 02:54:05,451 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e8b363c4e2d046ddb778cb9cc7a28c16, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:54:05,452 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting a18ce57c7ebf4ebb93a5c49722037094, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:54:05,452 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting a7c60767d91e4a3485f4c692c979a68a_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965244911 2023-04-03 02:54:05,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:05,453 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 000ede42e8f641cc9decbe0e68e614c3_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014844911 2023-04-03 02:54:05,454 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:05,456 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511206b82145f90aa4e26b266b11fd1a5c248 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511206b82145f90aa4e26b266b11fd1a5c248 2023-04-03 02:54:05,458 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b660269567544cf7848bbf2c6533193b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128b660269567544cf7848bbf2c6533193b 2023-04-03 02:54:05,460 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c93056671e8d43b89c11d8dbe1801e5d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c93056671e8d43b89c11d8dbe1801e5d 2023-04-03 02:54:05,460 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-04-03 02:54:05,461 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-03 02:54:05,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:05,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:05,462 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:54:05,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:54:05,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:05,463 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:05,467 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.7660 sec 2023-04-03 02:54:05,493 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5a1b3698dfeedaa8cd615262240ef68a#family1#compaction#23 average throughput is 6.77 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:05,609 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=266 (was 250) Potentially hanging thread: hconnection-0x9c0afc6-shared-pool-22 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57704 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57740 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/d21cdff62b2e495988cd575a7d3a690c block BP-2002395145-172.31.14.131-1680490416057:blk_1073741900_1076 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x7447ff9a-SendThread(127.0.0.1:52880) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57746 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57622 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-2002395145-172.31.14.131-1680490416057:blk_1073741900_1076 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x7447ff9a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272019329_15 at /127.0.0.1:57640 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57598 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-2002395145-172.31.14.131-1680490416057:blk_1073741900_1076, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57590 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57720 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:52880@0x7447ff9a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/523480629.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57586 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57780 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57716 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57668 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35658 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35854 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35804 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57566 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_272019329_15 at /127.0.0.1:57790 [Receiving block BP-2002395145-172.31.14.131-1680490416057:blk_1073741900_1076] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x9c0afc6-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x21ca7c4d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d0f40ca18b1a4018ab1f952685541ab6 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:35672 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-77168601_15 at /127.0.0.1:57696 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x9c0afc6-shared-pool-21 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=629 (was 596) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=521 (was 514) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=3766 (was 3870) 2023-04-03 02:54:05,627 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=266, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=521, ProcessCount=170, AvailableMemoryMB=3765 2023-04-03 02:54:05,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:54:05,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:05,634 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:05,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-04-03 02:54:05,635 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:05,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-03 02:54:05,637 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:05,638 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159 empty. 2023-04-03 02:54:05,638 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:05,639 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-04-03 02:54:05,669 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:05,671 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 91958613c4fcaae274fbfdf1b3c6a159, NAME => 'testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 91958613c4fcaae274fbfdf1b3c6a159, disabling compactions & flushes 2023-04-03 02:54:05,701 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. after waiting 0 ms 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:05,701 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:05,701 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:05,708 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:05,709 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490445709"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490445709"}]},"ts":"1680490445709"} 2023-04-03 02:54:05,711 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:54:05,713 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:05,713 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490445713"}]},"ts":"1680490445713"} 2023-04-03 02:54:05,715 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-04-03 02:54:05,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=91958613c4fcaae274fbfdf1b3c6a159, ASSIGN}] 2023-04-03 02:54:05,727 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=91958613c4fcaae274fbfdf1b3c6a159, ASSIGN 2023-04-03 02:54:05,729 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=91958613c4fcaae274fbfdf1b3c6a159, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:05,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-03 02:54:05,880 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=91958613c4fcaae274fbfdf1b3c6a159, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:05,880 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490445880"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490445880"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490445880"}]},"ts":"1680490445880"} 2023-04-03 02:54:05,883 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 91958613c4fcaae274fbfdf1b3c6a159, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:05,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-03 02:54:05,964 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/.tmp/family1/d21cdff62b2e495988cd575a7d3a690c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/d21cdff62b2e495988cd575a7d3a690c 2023-04-03 02:54:05,972 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5a1b3698dfeedaa8cd615262240ef68a/family1 of 5a1b3698dfeedaa8cd615262240ef68a into d21cdff62b2e495988cd575a7d3a690c(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:05,972 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:54:05,972 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1680490445450; duration=0sec 2023-04-03 02:54:05,972 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:06,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:06,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 91958613c4fcaae274fbfdf1b3c6a159, NAME => 'testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:06,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:06,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,041 INFO [StoreOpener-91958613c4fcaae274fbfdf1b3c6a159-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,043 DEBUG [StoreOpener-91958613c4fcaae274fbfdf1b3c6a159-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:06,043 DEBUG [StoreOpener-91958613c4fcaae274fbfdf1b3c6a159-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:06,043 INFO [StoreOpener-91958613c4fcaae274fbfdf1b3c6a159-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91958613c4fcaae274fbfdf1b3c6a159 columnFamilyName family1 2023-04-03 02:54:06,045 INFO [StoreOpener-91958613c4fcaae274fbfdf1b3c6a159-1] regionserver.HStore(310): Store=91958613c4fcaae274fbfdf1b3c6a159/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:06,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a18ce57c7ebf4ebb93a5c49722037094, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_] to archive 2023-04-03 02:54:06,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:54:06,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:06,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:06,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/e8b363c4e2d046ddb778cb9cc7a28c16 2023-04-03 02:54:06,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 91958613c4fcaae274fbfdf1b3c6a159; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60755588, jitterRate=-0.09467118978500366}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:06,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:06,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159., pid=25, masterSystemTime=1680490446035 2023-04-03 02:54:06,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a18ce57c7ebf4ebb93a5c49722037094 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a18ce57c7ebf4ebb93a5c49722037094 2023-04-03 02:54:06,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/a7c60767d91e4a3485f4c692c979a68a_SeqId_46_ 2023-04-03 02:54:06,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:06,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:06,057 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=91958613c4fcaae274fbfdf1b3c6a159, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:06,057 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490446057"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490446057"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490446057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490446057"}]},"ts":"1680490446057"} 2023-04-03 02:54:06,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/family1/000ede42e8f641cc9decbe0e68e614c3_SeqId_49_ 2023-04-03 02:54:06,061 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-04-03 02:54:06,061 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 91958613c4fcaae274fbfdf1b3c6a159, server=jenkins-hbase4.apache.org,45927,1680490420719 in 176 msec 2023-04-03 02:54:06,063 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-04-03 02:54:06,063 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=91958613c4fcaae274fbfdf1b3c6a159, ASSIGN in 338 msec 2023-04-03 02:54:06,063 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:06,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490446064"}]},"ts":"1680490446064"} 2023-04-03 02:54:06,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-04-03 02:54:06,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:06,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 440 msec 2023-04-03 02:54:06,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-03 02:54:06,240 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-04-03 02:54:06,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:06,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,263 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:06,263 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:06,263 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:06,263 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:06,264 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,264 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,265 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:06,265 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:06,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:06,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:06,265 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:06,265 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:06,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:06,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:06,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:06,268 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:06,268 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:06,268 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:06,269 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:54:06,689 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511308ae074b78e8d47cc8f248651cf5294dc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511308ae074b78e8d47cc8f248651cf5294dc 2023-04-03 02:54:07,093 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/f31aaafd09c44da79ebccc3424c86699 2023-04-03 02:54:07,099 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/f31aaafd09c44da79ebccc3424c86699 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/f31aaafd09c44da79ebccc3424c86699 2023-04-03 02:54:07,103 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/f31aaafd09c44da79ebccc3424c86699, entries=1000, sequenceid=5, filesize=165.1 K 2023-04-03 02:54:07,104 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 835ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:07,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:07,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:07,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:07,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:07,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:07,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:07,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:07,106 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,106 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:07,107 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,107 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:07,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:07,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45131132[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:07,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:07,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,111 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:07,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:07,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:07,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:07,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:07,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:07,112 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:07,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:07,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:07,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:07,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:07,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:07,114 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:07,114 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:07,114 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:07,114 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,114 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:07,115 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,115 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:07,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,116 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:07,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:07,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,117 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:07,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:07,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:07,119 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:07,120 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:07,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:07,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,120 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,120 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:07,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:07,220 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:07,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:07,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:07,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:07,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,227 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:07,227 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:07,227 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:07,227 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:07,228 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,228 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,230 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:07,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:07,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:07,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:07,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:07,232 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:07,232 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:07,232 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:07,232 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:07,232 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:07,232 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:07,232 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:07,232 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:54:07,648 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128cfa04ad3ac714f9983fb9114232acade to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128cfa04ad3ac714f9983fb9114232acade 2023-04-03 02:54:08,054 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/56ae5efe72fa40eea3216b0ad4b33311 2023-04-03 02:54:08,059 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/56ae5efe72fa40eea3216b0ad4b33311 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311 2023-04-03 02:54:08,064 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:54:08,065 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 833ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:08,065 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:08,065 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:08,065 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:08,066 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:08,066 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,066 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:08,066 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:08,068 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,068 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:08,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:08,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:08,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:08,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:08,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:08,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:08,070 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bc8ecbe[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:08,070 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:08,070 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,071 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:08,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:08,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:08,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:08,072 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:08,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:08,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:08,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:08,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:08,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:08,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:08,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:08,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:08,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:08,074 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:08,074 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:08,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:08,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,075 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:08,076 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,076 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:08,076 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:08,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:08,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:08,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:08,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,077 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:08,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:08,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:08,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:08,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:08,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,082 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,082 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:08,082 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:08,182 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:08,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:08,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:08,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:08,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,188 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:08,188 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:08,188 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:08,188 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:08,189 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,189 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,190 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:08,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:08,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:08,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:08,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:08,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:08,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,191 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,191 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:08,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:08,191 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:08,191 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:08,191 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:08,191 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:08,192 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:08,192 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:08,192 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:08,192 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:08,608 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201aa2a8b8e30e44f1a88ebc72f4ede9aed to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aa2a8b8e30e44f1a88ebc72f4ede9aed 2023-04-03 02:54:09,013 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/450d1143edbf42b8b69c14ff751507e3 2023-04-03 02:54:09,019 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/450d1143edbf42b8b69c14ff751507e3 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3 2023-04-03 02:54:09,024 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3, entries=1, sequenceid=13, filesize=4.9 K 2023-04-03 02:54:09,025 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 833ms, sequenceid=13, compaction requested=true 2023-04-03 02:54:09,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:09,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:09,025 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:09,025 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:09,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:09,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:09,027 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,027 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,027 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,027 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:09,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:09,029 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3352bea[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:09,029 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:09,029 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,030 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,030 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:09,030 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:09,030 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:09,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:09,033 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,033 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:09,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:09,035 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:09,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:09,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:09,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,035 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:09,036 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,036 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,036 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:09,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,037 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:09,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:09,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,038 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,042 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,042 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,042 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,042 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:09,043 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:09,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:09,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:09,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:09,042 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:09,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:09,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:09,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:09,043 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:09,144 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:09,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:09,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:09,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:09,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,151 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:09,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:09,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:09,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:09,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:09,153 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:09,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:09,154 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,154 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:09,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:09,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,154 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:09,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:09,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:09,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:09,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:09,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:09,155 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:09,155 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:09,155 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:09,569 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120553d250d8d6464620af2659daa0685a51 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120553d250d8d6464620af2659daa0685a51 2023-04-03 02:54:09,974 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/56827c2cbec447deab949d20fa5cbc18 2023-04-03 02:54:09,979 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/56827c2cbec447deab949d20fa5cbc18 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18 2023-04-03 02:54:09,984 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18, entries=1, sequenceid=17, filesize=4.9 K 2023-04-03 02:54:09,985 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 830ms, sequenceid=17, compaction requested=true 2023-04-03 02:54:09,985 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:09,985 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:09,985 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:09,987 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:09,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4651ce3b[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:09,988 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:09,988 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,989 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:09,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:09,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,990 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:09,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:09,991 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:09,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:09,993 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:09,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:09,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:09,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,993 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:09,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:09,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:09,996 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:09,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:09,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:09,999 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:09,999 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:10,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:09,999 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:10,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:10,000 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:10,100 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:10,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:10,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:10,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:10,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,106 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:10,106 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:10,106 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:10,106 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:10,107 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,107 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,108 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:10,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:10,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:10,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:10,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:10,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,109 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:10,109 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:10,109 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:10,110 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:10,110 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:10,110 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:10,110 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:10,110 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:10,523 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228dce782577966424f9aec2b75113eca1a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228dce782577966424f9aec2b75113eca1a 2023-04-03 02:54:10,929 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/603144894b4744de83cf801e2948a546 2023-04-03 02:54:10,935 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/603144894b4744de83cf801e2948a546 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546 2023-04-03 02:54:10,939 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546, entries=1, sequenceid=21, filesize=4.9 K 2023-04-03 02:54:10,940 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 830ms, sequenceid=21, compaction requested=true 2023-04-03 02:54:10,940 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:10,940 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:10,940 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:10,940 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:10,940 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,940 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:10,940 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:10,941 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,941 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:10,942 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,942 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:10,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:10,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b96dd42[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:10,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:10,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,950 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,950 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:10,950 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,950 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:10,950 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:10,950 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:10,951 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:10,951 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,951 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:10,952 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:10,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:10,954 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:10,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:10,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:10,954 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:10,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,954 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:10,955 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,955 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,955 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:10,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:10,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:10,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:10,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:10,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:10,956 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:10,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,957 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:10,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:10,959 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:10,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:10,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:10,959 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:10,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:10,960 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:10,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:11,060 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:11,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:11,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:11,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:11,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,066 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:11,067 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:11,067 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:11,067 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:11,068 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,068 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,069 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,069 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:11,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:11,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:11,069 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,070 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:11,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,070 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:11,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,070 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:11,071 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:11,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:11,071 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:11,071 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:11,071 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:11,485 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231e93ca867d4f24dd88a2bb7bd1799c715 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231e93ca867d4f24dd88a2bb7bd1799c715 2023-04-03 02:54:11,541 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:54:11,890 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/71549bc2c20c491783c460ad147e262b 2023-04-03 02:54:11,896 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/71549bc2c20c491783c460ad147e262b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b 2023-04-03 02:54:11,901 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b, entries=1, sequenceid=25, filesize=4.9 K 2023-04-03 02:54:11,902 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 831ms, sequenceid=25, compaction requested=true 2023-04-03 02:54:11,902 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:11,902 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:11,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:11,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:11,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:11,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:11,904 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:11,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:11,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:11,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57776f92[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:11,906 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:11,906 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,907 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:11,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:11,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:11,907 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,907 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:11,908 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:11,909 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:11,908 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,909 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:11,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:11,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:11,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:11,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:11,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:11,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:11,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:11,911 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:11,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:11,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:11,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,912 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:11,917 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,917 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:11,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:11,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,917 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:11,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:11,917 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:11,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:11,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,919 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:11,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:11,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:11,921 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:11,921 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:11,922 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:11,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:11,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:11,922 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:11,922 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:11,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:11,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:12,022 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:12,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:12,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:12,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:12,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,030 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:12,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:12,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:12,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:12,031 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,031 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,032 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:12,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:12,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:12,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:12,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:12,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,033 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:12,033 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:12,034 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:12,034 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:12,034 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:12,034 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:12,449 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c233e5ee04ae49ebb6c12c3a319c2936 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c233e5ee04ae49ebb6c12c3a319c2936 2023-04-03 02:54:12,854 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/bf4a40efd4c3477eb620a01f8683877a 2023-04-03 02:54:12,860 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/bf4a40efd4c3477eb620a01f8683877a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a 2023-04-03 02:54:12,864 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a, entries=1, sequenceid=29, filesize=4.9 K 2023-04-03 02:54:12,865 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 831ms, sequenceid=29, compaction requested=true 2023-04-03 02:54:12,865 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:12,865 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:12,865 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:12,865 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:12,865 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,865 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:12,865 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:12,868 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,868 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:12,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:12,868 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,868 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:12,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:12,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:12,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:12,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:12,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db774ed[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:12,869 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:12,870 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,870 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,871 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:12,871 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:12,871 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:12,871 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,871 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:12,872 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:12,872 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:12,872 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,872 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:12,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:12,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:12,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:12,874 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:12,874 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:12,874 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:12,874 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,874 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:12,876 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,876 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:12,876 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:12,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:12,876 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,877 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,880 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,880 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:12,880 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:12,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:12,880 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:12,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:12,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:12,880 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,880 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,881 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,981 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:12,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:12,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:12,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:12,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,988 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:12,988 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:12,988 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:12,988 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:12,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,989 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,990 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:12,990 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:12,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:12,990 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:12,990 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:12,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:12,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:12,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:12,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:12,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:12,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:12,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:12,991 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:12,992 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:12,992 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:12,992 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:13,411 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fe8503c0803f4b5cb7331423ef1529c2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103fe8503c0803f4b5cb7331423ef1529c2 2023-04-03 02:54:13,417 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/adbb7939871f4fd3af7cbb96d8d4c20f 2023-04-03 02:54:13,422 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/adbb7939871f4fd3af7cbb96d8d4c20f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f 2023-04-03 02:54:13,426 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f, entries=1, sequenceid=33, filesize=4.9 K 2023-04-03 02:54:13,427 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 435ms, sequenceid=33, compaction requested=true 2023-04-03 02:54:13,427 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:13,427 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:13,427 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:13,427 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:13,427 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,427 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:13,427 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:13,428 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,428 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:13,429 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,429 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@355f5d76[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:13,430 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:13,430 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,431 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,431 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:13,431 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:13,431 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:13,431 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,431 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:13,432 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,432 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:13,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:13,433 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:13,433 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:13,435 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:13,435 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:13,435 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:13,435 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,435 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:13,437 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,437 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:13,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:13,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,437 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:13,438 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,439 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:13,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:13,441 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:13,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:13,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:13,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:13,441 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:13,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:13,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:13,541 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:13,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:13,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:13,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:13,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,548 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:13,549 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:13,549 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:13,549 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:13,549 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,549 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:13,550 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:13,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:13,551 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:13,551 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:13,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:13,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:13,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:13,552 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:13,552 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:13,552 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:13,552 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:13,552 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:13,552 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:13,552 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:13,552 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:13,965 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090747a11186525d4870bf34f9a08f3e18e2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015090747a11186525d4870bf34f9a08f3e18e2 2023-04-03 02:54:14,371 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/5576202ce5ec451ca7e3ca67dd7d7de7 2023-04-03 02:54:14,377 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/5576202ce5ec451ca7e3ca67dd7d7de7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7 2023-04-03 02:54:14,383 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7, entries=1, sequenceid=37, filesize=4.9 K 2023-04-03 02:54:14,384 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 832ms, sequenceid=37, compaction requested=true 2023-04-03 02:54:14,384 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:14,384 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:14,384 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:14,384 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:14,384 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,384 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:14,384 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:14,387 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:14,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:14,387 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,388 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:14,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:14,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:14,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@188eea52[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:14,389 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:14,389 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,393 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,394 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,394 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:14,395 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:14,395 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:14,396 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:14,396 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:14,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:14,401 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:14,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:14,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:14,401 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,401 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:14,405 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:14,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,406 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:14,409 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:14,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:14,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:14,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:14,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:14,410 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:14,410 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:14,510 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:14,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:14,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:14,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:14,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,517 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:14,517 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:14,517 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:14,517 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:14,518 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,518 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,519 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:14,519 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:14,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:14,519 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:14,519 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:14,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:14,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,520 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:14,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:14,523 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:14,523 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:14,523 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:14,523 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. started... 2023-04-03 02:54:14,523 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 91958613c4fcaae274fbfdf1b3c6a159 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:14,936 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120f14954a03f5d4c63ba4d8b7260f2c469 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120f14954a03f5d4c63ba4d8b7260f2c469 2023-04-03 02:54:15,342 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/7021dc0913704479a14f0eb73e042f29 2023-04-03 02:54:15,348 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/7021dc0913704479a14f0eb73e042f29 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29 2023-04-03 02:54:15,353 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29, entries=1, sequenceid=41, filesize=4.9 K 2023-04-03 02:54:15,354 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 91958613c4fcaae274fbfdf1b3c6a159 in 831ms, sequenceid=41, compaction requested=true 2023-04-03 02:54:15,354 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:15,354 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:15,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:15,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:15,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:15,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:15,356 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:15,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:15,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:15,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:15,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:15,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74690d2e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:15,357 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:15,357 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,358 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,359 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:15,359 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:15,359 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:15,359 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,359 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:15,360 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,360 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:15,360 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:15,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:15,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:15,360 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:15,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:15,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:15,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:15,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:15,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:15,362 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:15,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:15,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:15,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,363 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:15,364 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,364 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:15,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:15,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,364 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:15,365 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:15,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,366 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:15,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:15,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:15,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-03 02:54:15,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:15,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:15,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:15,368 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:15,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:15,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:15,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:15,369 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,469 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:15,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:15,472 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-04-03 02:54:15,479 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:15,480 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:15,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:15,481 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491055481 2023-04-03 02:54:15,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:54:15,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-04-03 02:54:15,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:15,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x41259d9a to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:15,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@694ae96d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:15,527 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,065 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010359017262e952498ca5809e9fb8637c66 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010359017262e952498ca5809e9fb8637c66 2023-04-03 02:54:16,066 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:16,089 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:16,090 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35146, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:16,103 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:16,109 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33974, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:16,141 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1487952a016a4e608274440b8ba2359b first=Optional[r3] last=Optional[r6] 2023-04-03 02:54:16,142 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1487952a016a4e608274440b8ba2359b}] 2023-04-03 02:54:16,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1487952a016a4e608274440b8ba2359b for inclusion in 91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:16,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-03 02:54:16,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:16,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:16,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1487952a016a4e608274440b8ba2359b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rm6qjvgpmq6qs7m75i3cb5j3dspjpcfhidrl6e73glclj5sq7i1jtq8ui2s1padv/family1/1487952a016a4e608274440b8ba2359b 2023-04-03 02:54:16,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rm6qjvgpmq6qs7m75i3cb5j3dspjpcfhidrl6e73glclj5sq7i1jtq8ui2s1padv/family1/1487952a016a4e608274440b8ba2359b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_ 2023-04-03 02:54:16,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rm6qjvgpmq6qs7m75i3cb5j3dspjpcfhidrl6e73glclj5sq7i1jtq8ui2s1padv/family1/1487952a016a4e608274440b8ba2359b into 91958613c4fcaae274fbfdf1b3c6a159/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_ - updating store file list. 2023-04-03 02:54:16,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_ into 91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:16,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rm6qjvgpmq6qs7m75i3cb5j3dspjpcfhidrl6e73glclj5sq7i1jtq8ui2s1padv/family1/1487952a016a4e608274440b8ba2359b into 91958613c4fcaae274fbfdf1b3c6a159/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_) 2023-04-03 02:54:16,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rm6qjvgpmq6qs7m75i3cb5j3dspjpcfhidrl6e73glclj5sq7i1jtq8ui2s1padv/family1/1487952a016a4e608274440b8ba2359b 2023-04-03 02:54:16,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:54:16,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:54:16,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 91958613c4fcaae274fbfdf1b3c6a159/family1 is initiating minor compaction 2023-04-03 02:54:16,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@743fcf60; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:16,183 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 91958613c4fcaae274fbfdf1b3c6a159/family1 in testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:54:16,184 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp, totalSize=50.0 K 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 56ae5efe72fa40eea3216b0ad4b33311, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 450d1143edbf42b8b69c14ff751507e3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 56827c2cbec447deab949d20fa5cbc18, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 603144894b4744de83cf801e2948a546, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 71549bc2c20c491783c460ad147e262b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:54:16,184 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting bf4a40efd4c3477eb620a01f8683877a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:54:16,185 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting adbb7939871f4fd3af7cbb96d8d4c20f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:54:16,185 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 5576202ce5ec451ca7e3ca67dd7d7de7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:54:16,185 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7021dc0913704479a14f0eb73e042f29, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:54:16,185 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:54:16,188 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:16,189 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:16,191 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228dce782577966424f9aec2b75113eca1a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228dce782577966424f9aec2b75113eca1a 2023-04-03 02:54:16,201 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231e93ca867d4f24dd88a2bb7bd1799c715 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231e93ca867d4f24dd88a2bb7bd1799c715 2023-04-03 02:54:16,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c233e5ee04ae49ebb6c12c3a319c2936 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101c233e5ee04ae49ebb6c12c3a319c2936 2023-04-03 02:54:16,205 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103fe8503c0803f4b5cb7331423ef1529c2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103fe8503c0803f4b5cb7331423ef1529c2 2023-04-03 02:54:16,206 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-04-03 02:54:16,206 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-03 02:54:16,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,208 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,261 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 91958613c4fcaae274fbfdf1b3c6a159#family1#compaction#34 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:16,275 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f9359f5a87c3484ca3559f44a0b1eea2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205f9359f5a87c3484ca3559f44a0b1eea2 2023-04-03 02:54:16,276 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:16,311 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c6c5045744ed4817900f92c948c342eb first=Optional[r1] last=Optional[r2] 2023-04-03 02:54:16,312 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c6c5045744ed4817900f92c948c342eb}] 2023-04-03 02:54:16,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c6c5045744ed4817900f92c948c342eb for inclusion in 91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:16,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-04-03 02:54:16,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:16,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:16,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/c6c5045744ed4817900f92c948c342eb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__grhal8tp0jbv4fel9fbdc39ttfu5gk0i86v14e3vov83blv996etf8q9ma6rf2dn/family1/c6c5045744ed4817900f92c948c342eb 2023-04-03 02:54:16,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__grhal8tp0jbv4fel9fbdc39ttfu5gk0i86v14e3vov83blv996etf8q9ma6rf2dn/family1/c6c5045744ed4817900f92c948c342eb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/c0f16ad81ef247a9979c9263f9c198db_SeqId_46_ 2023-04-03 02:54:16,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__grhal8tp0jbv4fel9fbdc39ttfu5gk0i86v14e3vov83blv996etf8q9ma6rf2dn/family1/c6c5045744ed4817900f92c948c342eb into 91958613c4fcaae274fbfdf1b3c6a159/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/c0f16ad81ef247a9979c9263f9c198db_SeqId_46_ - updating store file list. 2023-04-03 02:54:16,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/c0f16ad81ef247a9979c9263f9c198db_SeqId_46_ into 91958613c4fcaae274fbfdf1b3c6a159/family1 2023-04-03 02:54:16,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__grhal8tp0jbv4fel9fbdc39ttfu5gk0i86v14e3vov83blv996etf8q9ma6rf2dn/family1/c6c5045744ed4817900f92c948c342eb into 91958613c4fcaae274fbfdf1b3c6a159/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/c0f16ad81ef247a9979c9263f9c198db_SeqId_46_) 2023-04-03 02:54:16,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithWeeklyPolicy__grhal8tp0jbv4fel9fbdc39ttfu5gk0i86v14e3vov83blv996etf8q9ma6rf2dn/family1/c6c5045744ed4817900f92c948c342eb 2023-04-03 02:54:16,333 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:16,334 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:16,336 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aa2a8b8e30e44f1a88ebc72f4ede9aed to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201aa2a8b8e30e44f1a88ebc72f4ede9aed 2023-04-03 02:54:16,338 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120553d250d8d6464620af2659daa0685a51 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120553d250d8d6464620af2659daa0685a51 2023-04-03 02:54:16,338 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-04-03 02:54:16,338 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-03 02:54:16,338 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:16,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:16,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:16,340 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:16,342 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 867 msec 2023-04-03 02:54:16,500 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=259 (was 266), OpenFileDescriptor=625 (was 629), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=475 (was 521), ProcessCount=170 (was 170), AvailableMemoryMB=3727 (was 3765) 2023-04-03 02:54:16,514 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=259, OpenFileDescriptor=625, MaxFileDescriptor=60000, SystemLoadAverage=475, ProcessCount=170, AvailableMemoryMB=3726 2023-04-03 02:54:16,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-03 02:54:16,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:16,522 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:16,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-04-03 02:54:16,523 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:16,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-03 02:54:16,526 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:16,527 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:16,527 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:16,528 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7 empty. 2023-04-03 02:54:16,528 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd empty. 2023-04-03 02:54:16,528 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7 empty. 2023-04-03 02:54:16,528 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:16,528 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:16,529 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:16,529 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-04-03 02:54:16,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-03 02:54:16,697 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/.tmp/family1/ee9369bd66ed4a47b0abd3bc93fddcd6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/ee9369bd66ed4a47b0abd3bc93fddcd6 2023-04-03 02:54:16,703 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 91958613c4fcaae274fbfdf1b3c6a159/family1 of 91958613c4fcaae274fbfdf1b3c6a159 into ee9369bd66ed4a47b0abd3bc93fddcd6(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:16,703 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:54:16,704 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1680490456183; duration=0sec 2023-04-03 02:54:16,704 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:16,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_] to archive 2023-04-03 02:54:16,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:16,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56ae5efe72fa40eea3216b0ad4b33311 2023-04-03 02:54:16,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/450d1143edbf42b8b69c14ff751507e3 2023-04-03 02:54:16,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/56827c2cbec447deab949d20fa5cbc18 2023-04-03 02:54:16,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/603144894b4744de83cf801e2948a546 2023-04-03 02:54:16,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/71549bc2c20c491783c460ad147e262b 2023-04-03 02:54:16,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/bf4a40efd4c3477eb620a01f8683877a 2023-04-03 02:54:16,763 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/adbb7939871f4fd3af7cbb96d8d4c20f 2023-04-03 02:54:16,767 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5576202ce5ec451ca7e3ca67dd7d7de7 2023-04-03 02:54:16,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/7021dc0913704479a14f0eb73e042f29 2023-04-03 02:54:16,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/family1/5876f5eacc744ee79fec662cd81d8fcf_SeqId_44_ 2023-04-03 02:54:16,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-03 02:54:16,948 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:16,951 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => 90ae7a86d67bedba5ae129851b513ac7, NAME => 'testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:16,952 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 97be2f1adfec8336e4da255ab5f7cafd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:16,952 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7bcfadc80838c8916c1e93da8ae801f7, NAME => 'testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:17,024 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,025 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing 90ae7a86d67bedba5ae129851b513ac7, disabling compactions & flushes 2023-04-03 02:54:17,025 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,025 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,025 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. after waiting 0 ms 2023-04-03 02:54:17,025 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,025 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,025 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 7bcfadc80838c8916c1e93da8ae801f7, disabling compactions & flushes 2023-04-03 02:54:17,037 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. after waiting 0 ms 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,037 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,037 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 7bcfadc80838c8916c1e93da8ae801f7: 2023-04-03 02:54:17,040 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,040 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 97be2f1adfec8336e4da255ab5f7cafd, disabling compactions & flushes 2023-04-03 02:54:17,041 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,041 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,041 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. after waiting 0 ms 2023-04-03 02:54:17,041 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,041 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,041 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 97be2f1adfec8336e4da255ab5f7cafd: 2023-04-03 02:54:17,044 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:17,045 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457044"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490457044"}]},"ts":"1680490457044"} 2023-04-03 02:54:17,045 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457044"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490457044"}]},"ts":"1680490457044"} 2023-04-03 02:54:17,045 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680490457044"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490457044"}]},"ts":"1680490457044"} 2023-04-03 02:54:17,048 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-03 02:54:17,050 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:17,050 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490457050"}]},"ts":"1680490457050"} 2023-04-03 02:54:17,052 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-04-03 02:54:17,055 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=7bcfadc80838c8916c1e93da8ae801f7, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=97be2f1adfec8336e4da255ab5f7cafd, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=90ae7a86d67bedba5ae129851b513ac7, ASSIGN}] 2023-04-03 02:54:17,061 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=7bcfadc80838c8916c1e93da8ae801f7, ASSIGN 2023-04-03 02:54:17,061 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=90ae7a86d67bedba5ae129851b513ac7, ASSIGN 2023-04-03 02:54:17,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=97be2f1adfec8336e4da255ab5f7cafd, ASSIGN 2023-04-03 02:54:17,062 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=7bcfadc80838c8916c1e93da8ae801f7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:17,063 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=90ae7a86d67bedba5ae129851b513ac7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:17,063 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=97be2f1adfec8336e4da255ab5f7cafd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:17,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-03 02:54:17,215 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=97be2f1adfec8336e4da255ab5f7cafd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,215 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680490457214"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490457214"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490457214"}]},"ts":"1680490457214"} 2023-04-03 02:54:17,215 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=90ae7a86d67bedba5ae129851b513ac7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,215 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457215"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490457215"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490457215"}]},"ts":"1680490457215"} 2023-04-03 02:54:17,216 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=7bcfadc80838c8916c1e93da8ae801f7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,216 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490457216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490457216"}]},"ts":"1680490457216"} 2023-04-03 02:54:17,218 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure 97be2f1adfec8336e4da255ab5f7cafd, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:17,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 7bcfadc80838c8916c1e93da8ae801f7, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:17,221 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure 90ae7a86d67bedba5ae129851b513ac7, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:17,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 90ae7a86d67bedba5ae129851b513ac7, NAME => 'testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.', STARTKEY => '2', ENDKEY => ''} 2023-04-03 02:54:17,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,383 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,386 DEBUG [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1 2023-04-03 02:54:17,386 DEBUG [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1 2023-04-03 02:54:17,386 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90ae7a86d67bedba5ae129851b513ac7 columnFamilyName family1 2023-04-03 02:54:17,389 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] regionserver.HStore(310): Store=90ae7a86d67bedba5ae129851b513ac7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,390 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,391 DEBUG [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family2 2023-04-03 02:54:17,391 DEBUG [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family2 2023-04-03 02:54:17,392 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90ae7a86d67bedba5ae129851b513ac7 columnFamilyName family2 2023-04-03 02:54:17,393 INFO [StoreOpener-90ae7a86d67bedba5ae129851b513ac7-1] regionserver.HStore(310): Store=90ae7a86d67bedba5ae129851b513ac7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:54:17,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:54:17,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:17,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 90ae7a86d67bedba5ae129851b513ac7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62879365, jitterRate=-0.06302444636821747}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:54:17,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:17,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7., pid=33, masterSystemTime=1680490457371 2023-04-03 02:54:17,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 97be2f1adfec8336e4da255ab5f7cafd, NAME => 'testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.', STARTKEY => '1', ENDKEY => '2'} 2023-04-03 02:54:17,407 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=90ae7a86d67bedba5ae129851b513ac7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,407 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457407"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490457407"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490457407"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490457407"}]},"ts":"1680490457407"} 2023-04-03 02:54:17,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,409 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,412 DEBUG [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/family1 2023-04-03 02:54:17,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-04-03 02:54:17,412 DEBUG [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/family1 2023-04-03 02:54:17,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure 90ae7a86d67bedba5ae129851b513ac7, server=jenkins-hbase4.apache.org,45927,1680490420719 in 188 msec 2023-04-03 02:54:17,413 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97be2f1adfec8336e4da255ab5f7cafd columnFamilyName family1 2023-04-03 02:54:17,413 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=90ae7a86d67bedba5ae129851b513ac7, ASSIGN in 357 msec 2023-04-03 02:54:17,414 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] regionserver.HStore(310): Store=97be2f1adfec8336e4da255ab5f7cafd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,414 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,418 DEBUG [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/family2 2023-04-03 02:54:17,418 DEBUG [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/family2 2023-04-03 02:54:17,419 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 97be2f1adfec8336e4da255ab5f7cafd columnFamilyName family2 2023-04-03 02:54:17,420 INFO [StoreOpener-97be2f1adfec8336e4da255ab5f7cafd-1] regionserver.HStore(310): Store=97be2f1adfec8336e4da255ab5f7cafd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:54:17,429 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:54:17,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:17,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 97be2f1adfec8336e4da255ab5f7cafd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74244545, jitterRate=0.10632993280887604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:54:17,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 97be2f1adfec8336e4da255ab5f7cafd: 2023-04-03 02:54:17,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd., pid=31, masterSystemTime=1680490457371 2023-04-03 02:54:17,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7bcfadc80838c8916c1e93da8ae801f7, NAME => 'testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.', STARTKEY => '', ENDKEY => '1'} 2023-04-03 02:54:17,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:17,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,437 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,439 DEBUG [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/family1 2023-04-03 02:54:17,439 DEBUG [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/family1 2023-04-03 02:54:17,439 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=97be2f1adfec8336e4da255ab5f7cafd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,440 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bcfadc80838c8916c1e93da8ae801f7 columnFamilyName family1 2023-04-03 02:54:17,440 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680490457439"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490457439"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490457439"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490457439"}]},"ts":"1680490457439"} 2023-04-03 02:54:17,441 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] regionserver.HStore(310): Store=7bcfadc80838c8916c1e93da8ae801f7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,441 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,443 DEBUG [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/family2 2023-04-03 02:54:17,443 DEBUG [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/family2 2023-04-03 02:54:17,443 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bcfadc80838c8916c1e93da8ae801f7 columnFamilyName family2 2023-04-03 02:54:17,444 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=29 2023-04-03 02:54:17,444 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure 97be2f1adfec8336e4da255ab5f7cafd, server=jenkins-hbase4.apache.org,45927,1680490420719 in 223 msec 2023-04-03 02:54:17,445 INFO [StoreOpener-7bcfadc80838c8916c1e93da8ae801f7-1] regionserver.HStore(310): Store=7bcfadc80838c8916c1e93da8ae801f7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:17,445 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=97be2f1adfec8336e4da255ab5f7cafd, ASSIGN in 389 msec 2023-04-03 02:54:17,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:54:17,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:54:17,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:17,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7bcfadc80838c8916c1e93da8ae801f7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71001074, jitterRate=0.057998448610305786}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:54:17,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7bcfadc80838c8916c1e93da8ae801f7: 2023-04-03 02:54:17,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7., pid=32, masterSystemTime=1680490457371 2023-04-03 02:54:17,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,456 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=7bcfadc80838c8916c1e93da8ae801f7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,457 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680490457456"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490457456"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490457456"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490457456"}]},"ts":"1680490457456"} 2023-04-03 02:54:17,460 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-04-03 02:54:17,460 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 7bcfadc80838c8916c1e93da8ae801f7, server=jenkins-hbase4.apache.org,45927,1680490420719 in 237 msec 2023-04-03 02:54:17,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-04-03 02:54:17,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=7bcfadc80838c8916c1e93da8ae801f7, ASSIGN in 405 msec 2023-04-03 02:54:17,464 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:17,464 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490457464"}]},"ts":"1680490457464"} 2023-04-03 02:54:17,466 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-04-03 02:54:17,468 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:17,470 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 951 msec 2023-04-03 02:54:17,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-03 02:54:17,633 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-04-03 02:54:17,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:17,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,648 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-04-03 02:54:17,648 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:17,648 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-04-03 02:54:17,648 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-04-03 02:54:17,648 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,649 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,650 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:17,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:17,650 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:17,650 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:17,650 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:17,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-04-03 02:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:17,652 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-04-03 02:54:17,656 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,656 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. started... 2023-04-03 02:54:17,665 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bcfadc80838c8916c1e93da8ae801f7: 2023-04-03 02:54:17,665 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:17,665 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:17,665 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. started... 2023-04-03 02:54:17,668 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:54:17,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:17,673 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 90ae7a86d67bedba5ae129851b513ac7 2/2 column families, dataSize=48 B heapSize=632 B 2023-04-03 02:54:17,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:54:17,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. started... 2023-04-03 02:54:17,675 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 97be2f1adfec8336e4da255ab5f7cafd: 2023-04-03 02:54:17,675 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:17,675 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:54:18,005 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-04-03 02:54:18,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403d0445074a03f44b2b5a1003e539b9e65 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403d0445074a03f44b2b5a1003e539b9e65 2023-04-03 02:54:18,511 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp/family1/b0dc3738203040c7babe6886153d4f48 2023-04-03 02:54:18,517 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp/family1/b0dc3738203040c7babe6886153d4f48 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48 2023-04-03 02:54:18,524 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48, entries=1, sequenceid=5, filesize=4.9 K 2023-04-03 02:54:18,525 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 90ae7a86d67bedba5ae129851b513ac7 in 852ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:18,526 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:18,526 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:18,526 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:54:18,526 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:54:18,526 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,527 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-04-03 02:54:18,527 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-04-03 02:54:18,529 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,529 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:18,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:18,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:18,530 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:18,530 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:18,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:18,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-04-03 02:54:18,531 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@323f0f52[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:18,531 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-04-03 02:54:18,531 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,532 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,532 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-04-03 02:54:18,532 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-04-03 02:54:18,532 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:18,533 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,533 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:18,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-04-03 02:54:18,534 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:18,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:18,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:18,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-04-03 02:54:18,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:18,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:18,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:18,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:18,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-04-03 02:54:18,540 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-04-03 02:54:18,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:18,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:18,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,540 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:18,541 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,541 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,541 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:18,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:18,542 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:18,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:18,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,543 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:18,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:18,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,550 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:18,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:18,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-04-03 02:54:18,551 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:18,550 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,551 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,551 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,551 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:18,551 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:18,651 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:18,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:18,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:18,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:18,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,659 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-04-03 02:54:18,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:18,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-04-03 02:54:18,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-04-03 02:54:18,660 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,660 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,663 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,663 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:18,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:18,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:18,663 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:18,663 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:18,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:18,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,664 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,672 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-04-03 02:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-04-03 02:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-04-03 02:54:18,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:18,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:54:18,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. started... 2023-04-03 02:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:18,672 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. started... 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. started... 2023-04-03 02:54:18,673 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 90ae7a86d67bedba5ae129851b513ac7 2/2 column families, dataSize=90 B heapSize=752 B 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bcfadc80838c8916c1e93da8ae801f7: 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 97be2f1adfec8336e4da255ab5f7cafd: 2023-04-03 02:54:18,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:54:18,673 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:54:18,673 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:54:19,090 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403a512637be6ca4dd48d86bd95d0ff5a0a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403a512637be6ca4dd48d86bd95d0ff5a0a 2023-04-03 02:54:19,099 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp/family1/039cf5ee64ff49809a5d5df882244fae 2023-04-03 02:54:19,106 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp/family1/039cf5ee64ff49809a5d5df882244fae as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae 2023-04-03 02:54:19,114 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae, entries=2, sequenceid=9, filesize=5.0 K 2023-04-03 02:54:19,116 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 90ae7a86d67bedba5ae129851b513ac7 in 443ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:19,118 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:19,118 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:19,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:54:19,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:54:19,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-04-03 02:54:19,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-04-03 02:54:19,121 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,121 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:19,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:19,122 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,122 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:19,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:19,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:19,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:19,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-04-03 02:54:19,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d44508e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:19,125 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-04-03 02:54:19,125 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,126 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-04-03 02:54:19,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-04-03 02:54:19,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:19,126 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,126 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:19,127 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-04-03 02:54:19,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:19,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-04-03 02:54:19,128 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:19,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:19,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:19,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:19,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:19,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-04-03 02:54:19,132 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-04-03 02:54:19,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:19,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:19,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,132 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:19,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:19,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:19,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:19,139 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:19,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:19,140 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:19,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:19,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:19,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:19,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:19,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:19,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-04-03 02:54:19,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:19,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-04-03 02:54:19,147 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:19,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:19,147 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:19,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:19,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:19,247 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:19,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:19,256 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-04-03 02:54:19,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-03 02:54:19,259 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-03 02:54:19,260 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491059260 2023-04-03 02:54:19,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:19,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:19,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-04-03 02:54:19,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-03 02:54:19,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x54806ed9 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:19,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a3ff4a5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:19,272 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230403 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-04-03 02:54:19,736 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403b319e1f130734f5d95f84145d56e3552 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403b319e1f130734f5d95f84145d56e3552 2023-04-03 02:54:19,737 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-03 02:54:19,752 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:19,754 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35158, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:19,771 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:19,774 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33980, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:19,790 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230403/family1/3bc6321f69194de487aec13f7333bcec first=Optional[k0] last=Optional[k1] 2023-04-03 02:54:19,791 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230403/family1/3bc6321f69194de487aec13f7333bcec}] 2023-04-03 02:54:19,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230403/family1/3bc6321f69194de487aec13f7333bcec for inclusion in 90ae7a86d67bedba5ae129851b513ac7/family1 2023-04-03 02:54:19,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-04-03 02:54:19,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-03 02:54:19,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:19,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230403/family1/3bc6321f69194de487aec13f7333bcec to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testScannerOnBulkLoadRefHFiles__vht9tjtiijsuardjnii77dfcthfm69kknhc05podsaqi9cld085g372j6mkqul7q/family1/3bc6321f69194de487aec13f7333bcec 2023-04-03 02:54:19,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testScannerOnBulkLoadRefHFiles__vht9tjtiijsuardjnii77dfcthfm69kknhc05podsaqi9cld085g372j6mkqul7q/family1/3bc6321f69194de487aec13f7333bcec as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_ 2023-04-03 02:54:19,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testScannerOnBulkLoadRefHFiles__vht9tjtiijsuardjnii77dfcthfm69kknhc05podsaqi9cld085g372j6mkqul7q/family1/3bc6321f69194de487aec13f7333bcec into 90ae7a86d67bedba5ae129851b513ac7/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_ - updating store file list. 2023-04-03 02:54:19,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_ into 90ae7a86d67bedba5ae129851b513ac7/family1 2023-04-03 02:54:19,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testScannerOnBulkLoadRefHFiles__vht9tjtiijsuardjnii77dfcthfm69kknhc05podsaqi9cld085g372j6mkqul7q/family1/3bc6321f69194de487aec13f7333bcec into 90ae7a86d67bedba5ae129851b513ac7/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_) 2023-04-03 02:54:19,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testScannerOnBulkLoadRefHFiles__vht9tjtiijsuardjnii77dfcthfm69kknhc05podsaqi9cld085g372j6mkqul7q/family1/3bc6321f69194de487aec13f7333bcec 2023-04-03 02:54:19,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:54:19,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:54:19,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 90ae7a86d67bedba5ae129851b513ac7/family1 is initiating minor compaction (all files) 2023-04-03 02:54:19,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4b6beb03; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:19,833 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 90ae7a86d67bedba5ae129851b513ac7/family1 in testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:54:19,833 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp, totalSize=15.1 K 2023-04-03 02:54:19,836 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting b0dc3738203040c7babe6886153d4f48, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490457633 2023-04-03 02:54:19,836 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-03 02:54:19,837 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting 039cf5ee64ff49809a5d5df882244fae, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490457633 2023-04-03 02:54:19,838 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:19,838 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting 664dc8884ca849999f04c28bd471ecca_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490457633 2023-04-03 02:54:19,840 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403a512637be6ca4dd48d86bd95d0ff5a0a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403a512637be6ca4dd48d86bd95d0ff5a0a 2023-04-03 02:54:19,847 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403d0445074a03f44b2b5a1003e539b9e65 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230403d0445074a03f44b2b5a1003e539b9e65 2023-04-03 02:54:19,847 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-04-03 02:54:19,847 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230403' 2023-04-03 02:54:19,848 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230403 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-04-03 02:54:19,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-03 02:54:19,848 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:19,850 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-03 02:54:19,852 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 593 msec 2023-04-03 02:54:19,873 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] throttle.PressureAwareThroughputController(145): 90ae7a86d67bedba5ae129851b513ac7#family1#compaction#37 average throughput is 0.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:19,887 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=267 (was 259) - Thread LEAK? -, OpenFileDescriptor=628 (was 625) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=475 (was 475), ProcessCount=170 (was 170), AvailableMemoryMB=3679 (was 3726) 2023-04-03 02:54:19,903 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=267, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=475, ProcessCount=170, AvailableMemoryMB=3678 2023-04-03 02:54:19,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:54:19,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:19,912 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:19,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-04-03 02:54:19,913 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:19,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-03 02:54:19,916 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:19,916 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11 empty. 2023-04-03 02:54:19,917 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:19,917 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-04-03 02:54:19,928 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/.tmp/family1/86e3ab712e614833b883e12d5f4fd629 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/86e3ab712e614833b883e12d5f4fd629 2023-04-03 02:54:19,935 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 90ae7a86d67bedba5ae129851b513ac7/family1 of 90ae7a86d67bedba5ae129851b513ac7 into 86e3ab712e614833b883e12d5f4fd629(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:19,935 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:54:19,935 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1680490459833; duration=0sec 2023-04-03 02:54:19,935 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:19,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_] to archive 2023-04-03 02:54:19,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:19,954 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:19,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/b0dc3738203040c7babe6886153d4f48 2023-04-03 02:54:19,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/039cf5ee64ff49809a5d5df882244fae 2023-04-03 02:54:19,957 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8d413c6a0cdb05564f66adc26a684a11, NAME => 'testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:19,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/family1/664dc8884ca849999f04c28bd471ecca_SeqId_12_ 2023-04-03 02:54:20,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-03 02:54:20,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 8d413c6a0cdb05564f66adc26a684a11, disabling compactions & flushes 2023-04-03 02:54:20,369 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. after waiting 0 ms 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,369 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,369 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:20,373 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:20,374 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490460374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490460374"}]},"ts":"1680490460374"} 2023-04-03 02:54:20,375 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:54:20,376 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:20,377 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490460376"}]},"ts":"1680490460376"} 2023-04-03 02:54:20,378 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-04-03 02:54:20,382 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8d413c6a0cdb05564f66adc26a684a11, ASSIGN}] 2023-04-03 02:54:20,384 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8d413c6a0cdb05564f66adc26a684a11, ASSIGN 2023-04-03 02:54:20,385 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8d413c6a0cdb05564f66adc26a684a11, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:20,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-03 02:54:20,536 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8d413c6a0cdb05564f66adc26a684a11, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:20,536 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490460536"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490460536"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490460536"}]},"ts":"1680490460536"} 2023-04-03 02:54:20,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 8d413c6a0cdb05564f66adc26a684a11, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:20,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8d413c6a0cdb05564f66adc26a684a11, NAME => 'testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,696 INFO [StoreOpener-8d413c6a0cdb05564f66adc26a684a11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,698 DEBUG [StoreOpener-8d413c6a0cdb05564f66adc26a684a11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:20,698 DEBUG [StoreOpener-8d413c6a0cdb05564f66adc26a684a11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:20,698 INFO [StoreOpener-8d413c6a0cdb05564f66adc26a684a11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8d413c6a0cdb05564f66adc26a684a11 columnFamilyName family1 2023-04-03 02:54:20,704 INFO [StoreOpener-8d413c6a0cdb05564f66adc26a684a11-1] regionserver.HStore(310): Store=8d413c6a0cdb05564f66adc26a684a11/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:20,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:54:20,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:20,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8d413c6a0cdb05564f66adc26a684a11; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61424546, jitterRate=-0.08470293879508972}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:20,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:20,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11., pid=37, masterSystemTime=1680490460690 2023-04-03 02:54:20,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:20,716 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8d413c6a0cdb05564f66adc26a684a11, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:20,716 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680490460716"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490460716"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490460716"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490460716"}]},"ts":"1680490460716"} 2023-04-03 02:54:20,720 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-04-03 02:54:20,720 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 8d413c6a0cdb05564f66adc26a684a11, server=jenkins-hbase4.apache.org,45927,1680490420719 in 180 msec 2023-04-03 02:54:20,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-04-03 02:54:20,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=8d413c6a0cdb05564f66adc26a684a11, ASSIGN in 338 msec 2023-04-03 02:54:20,725 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:20,725 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490460725"}]},"ts":"1680490460725"} 2023-04-03 02:54:20,727 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-04-03 02:54:20,731 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:20,733 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 823 msec 2023-04-03 02:54:21,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-03 02:54:21,020 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-04-03 02:54:21,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:21,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,080 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:21,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:21,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:21,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:21,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:21,083 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:21,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:21,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,084 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,084 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:21,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:21,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,092 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,092 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:21,093 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:21,093 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:21,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:21,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:21,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:21,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:54:21,544 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f6fea15bf59446f5af4cefe1eacc5156 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f6fea15bf59446f5af4cefe1eacc5156 2023-04-03 02:54:21,949 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/49aa3639a06449c0a8f0ed3170b8c94b 2023-04-03 02:54:21,955 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/49aa3639a06449c0a8f0ed3170b8c94b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b 2023-04-03 02:54:21,961 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b, entries=1000, sequenceid=5, filesize=165.1 K 2023-04-03 02:54:21,962 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 857ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:21,962 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:21,963 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:21,963 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:21,963 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:21,963 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:21,963 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,963 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:21,963 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:21,965 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:21,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:21,965 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:21,965 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,966 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:21,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:21,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:21,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:21,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1277e90e[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:21,967 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:21,967 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:21,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:21,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:21,970 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,970 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:21,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:21,971 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:21,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,971 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:21,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:21,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:21,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:21,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:21,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:21,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:21,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:21,975 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:21,975 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:21,975 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:21,975 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,975 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:21,979 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,979 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:21,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,979 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:21,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:21,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:21,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:21,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,981 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:21,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:21,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,982 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:21,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,985 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:21,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:21,985 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:21,985 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:21,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:21,985 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:21,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:21,985 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:21,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:21,986 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:21,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:21,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:22,086 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:22,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:22,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:22,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:22,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,094 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:22,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:22,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:22,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:22,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,097 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:22,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:22,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:22,098 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,098 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:22,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,099 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,099 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:22,099 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,099 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:22,099 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:22,099 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:22,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:22,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:22,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:22,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:22,104 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:54:22,439 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:22,440 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54806ed9 to 127.0.0.1:52880 2023-04-03 02:54:22,440 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:22,441 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:22,441 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x41259d9a to 127.0.0.1:52880 2023-04-03 02:54:22,441 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:22,442 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:22,442 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a57b8a1 to 127.0.0.1:52880 2023-04-03 02:54:22,442 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:22,442 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:22,442 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7447ff9a to 127.0.0.1:52880 2023-04-03 02:54:22,442 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:22,443 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:22,444 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b052bf3 to 127.0.0.1:52880 2023-04-03 02:54:22,444 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:22,525 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d704f183e0714df9abcae5b60a525e79 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128d704f183e0714df9abcae5b60a525e79 2023-04-03 02:54:22,933 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/c165477f928441ffb573db91cc1cb2a6 2023-04-03 02:54:22,939 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/c165477f928441ffb573db91cc1cb2a6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6 2023-04-03 02:54:22,944 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:54:22,945 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 841ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:22,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:22,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:22,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:22,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:22,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:22,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:22,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:22,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:22,947 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:22,948 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,948 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:22,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:22,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:22,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:22,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e4b9613[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:22,949 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:22,949 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,950 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:54:22,954 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,954 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:22,954 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:22,954 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:22,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:22,955 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:22,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:22,955 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:22,956 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:22,956 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:22,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:22,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:22,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:22,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:22,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:22,959 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:22,959 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:22,959 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:22,959 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,959 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:22,961 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,961 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:22,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,961 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:22,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,964 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,965 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:22,969 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:22,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:22,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:22,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:22,970 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:22,970 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:22,970 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:22,970 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:23,070 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:23,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:23,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:23,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:23,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,078 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:23,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:23,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:23,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:23,079 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,079 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,080 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,080 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:23,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:23,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:23,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,080 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:23,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:23,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:23,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:23,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:23,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:23,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:23,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:23,082 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:23,082 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:23,082 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:23,497 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201bc425157064b4984913500269a25156a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201bc425157064b4984913500269a25156a 2023-04-03 02:54:23,904 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/46a447b3d1594e6099a0960be7fa1b6e 2023-04-03 02:54:23,910 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/46a447b3d1594e6099a0960be7fa1b6e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e 2023-04-03 02:54:23,917 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e, entries=1, sequenceid=13, filesize=4.9 K 2023-04-03 02:54:23,918 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 836ms, sequenceid=13, compaction requested=true 2023-04-03 02:54:23,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:23,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:23,922 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:23,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:23,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:23,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:23,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:23,924 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64c44917[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:23,924 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:23,924 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,925 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:23,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:23,926 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,926 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:23,927 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:23,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:23,930 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:23,930 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:23,930 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:23,930 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,930 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:23,932 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:23,932 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,932 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:23,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:23,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:23,933 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,935 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:23,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,940 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:23,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:23,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:23,941 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:23,941 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:23,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:23,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:23,941 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:23,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:23,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:24,041 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:24,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:24,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:24,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:24,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,064 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:24,064 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,064 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:24,064 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:24,065 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,065 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,066 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,067 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,067 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:24,067 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:24,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:24,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:24,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:24,073 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:24,073 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:24,073 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:24,104 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e788734c288f442ab9794726b062c866 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e788734c288f442ab9794726b062c866 2023-04-03 02:54:24,115 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/e4cf1fe22f8541ef91ece11a2dfce263 2023-04-03 02:54:24,121 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/e4cf1fe22f8541ef91ece11a2dfce263 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263 2023-04-03 02:54:24,130 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263, entries=1, sequenceid=17, filesize=4.9 K 2023-04-03 02:54:24,132 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 59ms, sequenceid=17, compaction requested=true 2023-04-03 02:54:24,133 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:24,133 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:24,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:24,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:24,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:24,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:24,135 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,135 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:24,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:24,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3bdbb521[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:24,138 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:24,138 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,139 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:24,139 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,139 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:24,140 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:24,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:24,143 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:24,143 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:24,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:24,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:24,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,143 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:24,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,147 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,147 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,147 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,149 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:24,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:24,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:24,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:24,153 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:24,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:24,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:24,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:24,154 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:24,154 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:24,254 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:24,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:24,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:24,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:24,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,280 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:24,280 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,280 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:24,280 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:24,282 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,282 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,286 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:24,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:24,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:24,287 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,287 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:24,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:24,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:24,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,292 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:24,292 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,292 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:24,292 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:24,292 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:24,293 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:24,294 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:24,294 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:24,318 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280bbd6ffc7be14af29ffc066d9545cefc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512280bbd6ffc7be14af29ffc066d9545cefc 2023-04-03 02:54:24,763 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/53151c7fb4cb40ab8405f7c4d3877b41 2023-04-03 02:54:24,769 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/53151c7fb4cb40ab8405f7c4d3877b41 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41 2023-04-03 02:54:24,774 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41, entries=1, sequenceid=21, filesize=4.9 K 2023-04-03 02:54:24,775 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 481ms, sequenceid=21, compaction requested=true 2023-04-03 02:54:24,775 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:24,775 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:24,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:24,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:24,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:24,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:24,777 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,777 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,777 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,777 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@164914cd[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:24,779 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:24,779 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,780 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:24,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:24,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:24,780 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,780 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:24,781 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:24,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:24,784 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:24,784 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:24,784 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:24,784 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,784 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:24,791 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,791 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:24,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:24,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,791 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:24,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:24,792 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,794 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:24,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,796 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,796 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:24,796 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,796 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:24,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,801 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:24,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:24,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:24,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:24,801 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:24,902 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:24,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:24,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:24,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:24,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,910 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:24,910 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,910 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:24,910 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:24,911 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,911 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,915 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:24,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:24,916 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,916 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:24,916 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,917 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,917 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,918 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:24,918 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:24,918 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:24,918 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:24,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:24,919 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:24,927 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:24,927 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:24,927 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:25,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123149fc9c219b6d431aabfbf2936f4c444d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123149fc9c219b6d431aabfbf2936f4c444d 2023-04-03 02:54:25,357 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/506f34c210aa49a28f9b552b695ea9f7 2023-04-03 02:54:25,363 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/506f34c210aa49a28f9b552b695ea9f7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7 2023-04-03 02:54:25,368 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7, entries=1, sequenceid=25, filesize=4.9 K 2023-04-03 02:54:25,369 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 442ms, sequenceid=25, compaction requested=true 2023-04-03 02:54:25,370 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:25,370 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:25,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:25,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:25,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:25,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:25,373 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,373 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:25,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:25,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64c0f319[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:25,375 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:25,375 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,376 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,376 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:25,376 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(162): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,376 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,376 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:25,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:25,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:25,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:25,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:25,378 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:25,383 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:25,383 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:25,383 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:25,383 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,383 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:25,385 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,392 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,392 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:25,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,392 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:25,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,395 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:25,398 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,402 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:25,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:25,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:25,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:25,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:25,403 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:25,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:25,403 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:25,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:25,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:25,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:25,503 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:25,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:25,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:25,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:25,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,513 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:25,513 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:25,513 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:25,513 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:25,514 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,514 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,515 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:25,515 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:25,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:25,515 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,516 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:25,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:25,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,519 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:25,519 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,519 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:25,519 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:25,519 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:25,519 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:25,519 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:25,519 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:25,519 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:25,519 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:25,970 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101abfa0b77e24a47aa8f50eba6df479e18 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101abfa0b77e24a47aa8f50eba6df479e18 2023-04-03 02:54:25,976 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/c902d2d333e747d4943f6e2d835670d9 2023-04-03 02:54:25,981 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/c902d2d333e747d4943f6e2d835670d9 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9 2023-04-03 02:54:25,985 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9, entries=1, sequenceid=29, filesize=4.9 K 2023-04-03 02:54:25,986 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 467ms, sequenceid=29, compaction requested=true 2023-04-03 02:54:25,986 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:25,986 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:25,989 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@228a9267[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:25,991 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:25,991 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,992 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,992 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:25,993 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:25,993 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:25,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,993 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:25,994 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,994 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,994 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:25,994 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:25,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:25,997 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:25,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:25,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:25,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,997 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:25,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:25,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:25,999 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:25,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,001 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,007 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:26,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:26,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:26,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:26,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:26,007 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,008 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,008 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,008 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:26,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:26,008 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:26,008 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:26,108 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:26,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:26,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:26,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:26,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,115 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:26,115 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:26,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:26,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:26,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,117 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:26,117 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:26,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:26,119 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:26,119 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:26,119 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:26,536 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033ce20db33f2c4df9b309a363d92c13c5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033ce20db33f2c4df9b309a363d92c13c5 2023-04-03 02:54:26,550 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/14a788d3b208430589bd72e3c2a92e4a 2023-04-03 02:54:26,557 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/14a788d3b208430589bd72e3c2a92e4a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a 2023-04-03 02:54:26,562 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a, entries=1, sequenceid=33, filesize=4.9 K 2023-04-03 02:54:26,563 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 444ms, sequenceid=33, compaction requested=true 2023-04-03 02:54:26,563 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:26,563 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:26,563 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:26,565 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,565 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:26,566 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,566 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:26,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:26,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@441e59aa[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:26,568 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:26,568 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,569 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,570 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:26,570 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:26,570 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:26,570 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,570 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:26,572 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:26,572 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:26,572 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,572 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:26,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:26,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:26,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:26,576 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:26,576 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:26,576 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:26,576 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,576 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:26,578 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,578 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,578 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:26,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:26,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:26,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,581 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,583 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:26,583 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:26,584 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:26,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:26,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:26,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:26,588 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:26,588 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:26,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:26,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:26,589 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-03 02:54:26,688 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:26,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:26,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:26,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:26,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,696 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:26,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:26,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:26,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:26,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,698 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,698 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:26,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:26,699 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:26,699 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,700 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:26,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:26,701 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:26,701 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:26,701 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:26,720 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070b527c04dd9d4dfca354a7a31d7ce2a0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509070b527c04dd9d4dfca354a7a31d7ce2a0 2023-04-03 02:54:27,130 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/d10755d111e24a069a8d27d43f01f5bd 2023-04-03 02:54:27,136 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/d10755d111e24a069a8d27d43f01f5bd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd 2023-04-03 02:54:27,140 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd, entries=1, sequenceid=37, filesize=4.9 K 2023-04-03 02:54:27,141 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 440ms, sequenceid=37, compaction requested=true 2023-04-03 02:54:27,141 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:27,141 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:27,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:27,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,144 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:27,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,144 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a2fd298[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:27,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:27,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,147 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,147 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:27,153 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:27,153 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:27,155 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:27,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:27,155 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:27,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:27,158 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:27,158 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:27,158 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:27,158 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,158 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:27,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:27,159 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:27,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:27,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:27,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:27,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,162 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:27,163 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:27,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:27,168 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:27,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:27,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:27,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,168 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,168 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,168 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,168 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,167 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:27,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:27,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:27,268 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:27,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:27,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:27,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:27,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,276 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:27,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:27,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-03 02:54:27,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-03 02:54:27,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,277 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:27,277 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:27,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:27,278 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:27,278 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:27,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-03 02:54:27,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-03 02:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-03 02:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:27,279 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:27,279 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:27,279 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. started... 2023-04-03 02:54:27,279 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8d413c6a0cdb05564f66adc26a684a11 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:27,694 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e27320ab932845b6b31b28c7a53f752c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120e27320ab932845b6b31b28c7a53f752c 2023-04-03 02:54:28,114 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/fd68a015c493450393738754e1b0eb08 2023-04-03 02:54:28,120 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/fd68a015c493450393738754e1b0eb08 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08 2023-04-03 02:54:28,124 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08, entries=1, sequenceid=41, filesize=4.9 K 2023-04-03 02:54:28,125 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8d413c6a0cdb05564f66adc26a684a11 in 846ms, sequenceid=41, compaction requested=true 2023-04-03 02:54:28,125 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:28,125 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:28,125 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:28,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:28,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-03 02:54:28,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-03 02:54:28,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:28,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:28,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-03 02:54:28,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7994b2ff[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:28,129 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-03 02:54:28,130 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,131 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-03 02:54:28,131 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-03 02:54:28,131 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:28,131 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,131 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:28,133 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-03 02:54:28,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:28,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:28,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:28,133 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-03 02:54:28,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:28,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:28,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:28,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:28,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:54:28,136 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-03 02:54:28,136 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:28,136 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:28,136 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,136 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:28,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:28,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:28,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:28,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:28,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,139 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:28,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:28,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:28,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,140 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:28,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,145 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:28,145 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,145 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:28,145 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:28,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:28,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:28,145 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-03 02:54:28,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:28,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:28,146 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:28,246 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:28,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:28,248 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-04-03 02:54:28,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:28,251 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:28,252 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491068252 2023-04-03 02:54:28,253 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:28,254 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:28,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-04-03 02:54:28,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:28,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x3e802a7d to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:28,268 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b8f7bd4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:28,268 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,128 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103384926b41b9e49269fa990e243368efa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103384926b41b9e49269fa990e243368efa 2023-04-03 02:54:29,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:29,148 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:29,149 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33370, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:29,160 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:29,161 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55952, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:29,170 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0797dec1487a4a99b14ca4241c205369 first=Optional[r3] last=Optional[r6] 2023-04-03 02:54:29,171 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0797dec1487a4a99b14ca4241c205369}] 2023-04-03 02:54:29,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0797dec1487a4a99b14ca4241c205369 for inclusion in 8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:29,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-03 02:54:29,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:29,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:29,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0797dec1487a4a99b14ca4241c205369 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__d5udhgjfbvpqi9v0eslf66aq07dni732slvnv16kiubmllf6pck0l0004evc8p29/family1/0797dec1487a4a99b14ca4241c205369 2023-04-03 02:54:29,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__d5udhgjfbvpqi9v0eslf66aq07dni732slvnv16kiubmllf6pck0l0004evc8p29/family1/0797dec1487a4a99b14ca4241c205369 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_ 2023-04-03 02:54:29,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__d5udhgjfbvpqi9v0eslf66aq07dni732slvnv16kiubmllf6pck0l0004evc8p29/family1/0797dec1487a4a99b14ca4241c205369 into 8d413c6a0cdb05564f66adc26a684a11/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_ - updating store file list. 2023-04-03 02:54:29,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_ into 8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:29,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__d5udhgjfbvpqi9v0eslf66aq07dni732slvnv16kiubmllf6pck0l0004evc8p29/family1/0797dec1487a4a99b14ca4241c205369 into 8d413c6a0cdb05564f66adc26a684a11/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_) 2023-04-03 02:54:29,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__d5udhgjfbvpqi9v0eslf66aq07dni732slvnv16kiubmllf6pck0l0004evc8p29/family1/0797dec1487a4a99b14ca4241c205369 2023-04-03 02:54:29,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:54:29,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:54:29,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 8d413c6a0cdb05564f66adc26a684a11/family1 is initiating minor compaction 2023-04-03 02:54:29,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@33307284; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:29,190 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8d413c6a0cdb05564f66adc26a684a11/family1 in testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:29,191 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp, totalSize=50.0 K 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting c165477f928441ffb573db91cc1cb2a6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 46a447b3d1594e6099a0960be7fa1b6e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e4cf1fe22f8541ef91ece11a2dfce263, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 53151c7fb4cb40ab8405f7c4d3877b41, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 506f34c210aa49a28f9b552b695ea9f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting c902d2d333e747d4943f6e2d835670d9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 14a788d3b208430589bd72e3c2a92e4a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting d10755d111e24a069a8d27d43f01f5bd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting fd68a015c493450393738754e1b0eb08, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:54:29,191 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f28e60b68d154e61aa03a1f171cbec58_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:54:29,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:29,193 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:29,195 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512280bbd6ffc7be14af29ffc066d9545cefc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512280bbd6ffc7be14af29ffc066d9545cefc 2023-04-03 02:54:29,197 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123149fc9c219b6d431aabfbf2936f4c444d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123149fc9c219b6d431aabfbf2936f4c444d 2023-04-03 02:54:29,199 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101abfa0b77e24a47aa8f50eba6df479e18 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101abfa0b77e24a47aa8f50eba6df479e18 2023-04-03 02:54:29,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033ce20db33f2c4df9b309a363d92c13c5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033ce20db33f2c4df9b309a363d92c13c5 2023-04-03 02:54:29,200 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-04-03 02:54:29,200 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,240 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8d413c6a0cdb05564f66adc26a684a11#family1#compaction#48 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:29,657 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/d849d4eddb444a318bfa29802a92447c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d849d4eddb444a318bfa29802a92447c 2023-04-03 02:54:29,663 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8d413c6a0cdb05564f66adc26a684a11/family1 of 8d413c6a0cdb05564f66adc26a684a11 into d849d4eddb444a318bfa29802a92447c(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:29,663 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:29,663 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1680490469190; duration=0sec 2023-04-03 02:54:29,664 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:29,681 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ff33e4edec0f4a46a119a879cf12b413 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205ff33e4edec0f4a46a119a879cf12b413 2023-04-03 02:54:29,682 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:29,717 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8c485d79e4254ee6a4d5a6e07b84dadf first=Optional[r00] last=Optional[r2] 2023-04-03 02:54:29,718 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8c485d79e4254ee6a4d5a6e07b84dadf}] 2023-04-03 02:54:29,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8c485d79e4254ee6a4d5a6e07b84dadf for inclusion in 8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:29,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-03 02:54:29,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:29,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:29,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8c485d79e4254ee6a4d5a6e07b84dadf to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mhb7s2luq17qi6k2ts5bjj4q8i6m9p3kk925feumo8hfo1sccl76660t8ueneh86/family1/8c485d79e4254ee6a4d5a6e07b84dadf 2023-04-03 02:54:29,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mhb7s2luq17qi6k2ts5bjj4q8i6m9p3kk925feumo8hfo1sccl76660t8ueneh86/family1/8c485d79e4254ee6a4d5a6e07b84dadf as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_ 2023-04-03 02:54:29,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mhb7s2luq17qi6k2ts5bjj4q8i6m9p3kk925feumo8hfo1sccl76660t8ueneh86/family1/8c485d79e4254ee6a4d5a6e07b84dadf into 8d413c6a0cdb05564f66adc26a684a11/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_ - updating store file list. 2023-04-03 02:54:29,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_ into 8d413c6a0cdb05564f66adc26a684a11/family1 2023-04-03 02:54:29,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mhb7s2luq17qi6k2ts5bjj4q8i6m9p3kk925feumo8hfo1sccl76660t8ueneh86/family1/8c485d79e4254ee6a4d5a6e07b84dadf into 8d413c6a0cdb05564f66adc26a684a11/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_) 2023-04-03 02:54:29,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyPolicy__mhb7s2luq17qi6k2ts5bjj4q8i6m9p3kk925feumo8hfo1sccl76660t8ueneh86/family1/8c485d79e4254ee6a4d5a6e07b84dadf 2023-04-03 02:54:29,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:54:29,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:54:29,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 8d413c6a0cdb05564f66adc26a684a11/family1 is initiating minor compaction (all files) 2023-04-03 02:54:29,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@67975621; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:29,737 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8d413c6a0cdb05564f66adc26a684a11/family1 in testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:54:29,737 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d849d4eddb444a318bfa29802a92447c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp, totalSize=337.0 K 2023-04-03 02:54:29,738 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 49aa3639a06449c0a8f0ed3170b8c94b, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:54:29,738 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting d849d4eddb444a318bfa29802a92447c, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:54:29,738 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878844911 2023-04-03 02:54:29,741 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:29,747 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:29,754 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f6fea15bf59446f5af4cefe1eacc5156 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f6fea15bf59446f5af4cefe1eacc5156 2023-04-03 02:54:29,755 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_] to archive 2023-04-03 02:54:29,756 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201bc425157064b4984913500269a25156a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201bc425157064b4984913500269a25156a 2023-04-03 02:54:29,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:29,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e788734c288f442ab9794726b062c866 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e788734c288f442ab9794726b062c866 2023-04-03 02:54:29,758 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-04-03 02:54:29,758 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-03 02:54:29,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c165477f928441ffb573db91cc1cb2a6 2023-04-03 02:54:29,759 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-03 02:54:29,759 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-03 02:54:29,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:29,761 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:54:29,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/46a447b3d1594e6099a0960be7fa1b6e 2023-04-03 02:54:29,763 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.5120 sec 2023-04-03 02:54:29,765 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/e4cf1fe22f8541ef91ece11a2dfce263 2023-04-03 02:54:29,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/53151c7fb4cb40ab8405f7c4d3877b41 2023-04-03 02:54:29,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/506f34c210aa49a28f9b552b695ea9f7 2023-04-03 02:54:29,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c902d2d333e747d4943f6e2d835670d9 2023-04-03 02:54:29,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/14a788d3b208430589bd72e3c2a92e4a 2023-04-03 02:54:29,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d10755d111e24a069a8d27d43f01f5bd 2023-04-03 02:54:29,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/fd68a015c493450393738754e1b0eb08 2023-04-03 02:54:29,778 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8d413c6a0cdb05564f66adc26a684a11#family1#compaction#49 average throughput is 9.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:29,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/f28e60b68d154e61aa03a1f171cbec58_SeqId_44_ 2023-04-03 02:54:29,837 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/.tmp/family1/c0e30b8d82c1452188239b512631fa3a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/c0e30b8d82c1452188239b512631fa3a 2023-04-03 02:54:29,846 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8d413c6a0cdb05564f66adc26a684a11/family1 of 8d413c6a0cdb05564f66adc26a684a11 into c0e30b8d82c1452188239b512631fa3a(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:29,847 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:54:29,847 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1680490469737; duration=0sec 2023-04-03 02:54:29,847 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:29,908 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=250 (was 267), OpenFileDescriptor=603 (was 626), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=484 (was 475) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=3591 (was 3678) 2023-04-03 02:54:29,921 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=250, OpenFileDescriptor=603, MaxFileDescriptor=60000, SystemLoadAverage=484, ProcessCount=170, AvailableMemoryMB=3590 2023-04-03 02:54:29,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:54:29,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:29,926 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:29,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-04-03 02:54:29,926 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:29,928 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:29,928 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 empty. 2023-04-03 02:54:29,931 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:29,931 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-04-03 02:54:29,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-03 02:54:29,942 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:29,943 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 90d3745b2d090a6eb1d4604d211e6a26, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:29,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d849d4eddb444a318bfa29802a92447c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_] to archive 2023-04-03 02:54:29,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:29,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/49aa3639a06449c0a8f0ed3170b8c94b 2023-04-03 02:54:29,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d849d4eddb444a318bfa29802a92447c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/d849d4eddb444a318bfa29802a92447c 2023-04-03 02:54:29,959 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/family1/762280b8dbdc4145833f083a1a3d7bd6_SeqId_47_ 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 90d3745b2d090a6eb1d4604d211e6a26, disabling compactions & flushes 2023-04-03 02:54:29,979 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. after waiting 0 ms 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:29,979 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:29,979 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:29,982 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:29,983 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490469982"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490469982"}]},"ts":"1680490469982"} 2023-04-03 02:54:29,984 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:54:29,985 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:29,985 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490469985"}]},"ts":"1680490469985"} 2023-04-03 02:54:29,987 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-04-03 02:54:29,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, ASSIGN}] 2023-04-03 02:54:29,992 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, ASSIGN 2023-04-03 02:54:29,993 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:30,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-03 02:54:30,144 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:30,145 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490470144"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490470144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490470144"}]},"ts":"1680490470144"} 2023-04-03 02:54:30,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:30,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-03 02:54:30,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 90d3745b2d090a6eb1d4604d211e6a26, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,304 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,305 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:30,305 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:30,306 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90d3745b2d090a6eb1d4604d211e6a26 columnFamilyName family1 2023-04-03 02:54:30,307 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(310): Store=90d3745b2d090a6eb1d4604d211e6a26/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:30,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:30,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:30,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 90d3745b2d090a6eb1d4604d211e6a26; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65271940, jitterRate=-0.027372300624847412}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:30,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:30,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., pid=41, masterSystemTime=1680490470299 2023-04-03 02:54:30,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:30,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:30,317 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:30,317 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490470317"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490470317"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490470317"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490470317"}]},"ts":"1680490470317"} 2023-04-03 02:54:30,320 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-04-03 02:54:30,320 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719 in 171 msec 2023-04-03 02:54:30,322 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-04-03 02:54:30,322 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, ASSIGN in 330 msec 2023-04-03 02:54:30,322 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:30,322 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490470322"}]},"ts":"1680490470322"} 2023-04-03 02:54:30,324 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-04-03 02:54:30,326 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:30,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 403 msec 2023-04-03 02:54:30,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-03 02:54:30,536 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-04-03 02:54:30,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:30,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,568 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:30,568 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:30,568 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:30,569 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:30,569 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,569 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,571 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:30,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:30,571 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:30,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:30,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:30,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:30,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:30,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,580 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:30,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:30,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:30,580 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:30,586 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:30,586 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:30,586 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:30,586 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:30,587 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:54:31,024 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bab02b7c34c044e6b70e4a310fdf29eb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130bab02b7c34c044e6b70e4a310fdf29eb 2023-04-03 02:54:31,429 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/a9bb1575299d4d3d86804e49667edc1c 2023-04-03 02:54:31,434 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/a9bb1575299d4d3d86804e49667edc1c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c 2023-04-03 02:54:31,438 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c, entries=1000, sequenceid=5, filesize=191.4 K 2023-04-03 02:54:31,439 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 852ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:31,439 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:31,440 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:31,440 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:31,440 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:31,440 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:31,440 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,440 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:31,440 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:31,442 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,442 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,442 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,442 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:31,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ffb1e00[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:31,443 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:31,443 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,447 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:31,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:31,447 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:31,447 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,447 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:31,448 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,448 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:31,449 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,449 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:31,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:31,451 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:31,451 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:31,451 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:31,451 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,451 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:31,454 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,454 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,454 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:31,454 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:31,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,455 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,457 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:31,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:31,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,458 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:31,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:31,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:31,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:31,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:31,458 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:31,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:31,559 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:31,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:31,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:31,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:31,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,572 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:31,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:31,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:31,572 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:31,573 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,573 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,574 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,574 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:31,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:31,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:31,574 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,574 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:31,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:31,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,575 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,576 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:31,576 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:31,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:31,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:31,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:31,576 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:31,576 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:31,576 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:31,576 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:54:31,598 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112807464096a7814c58ad1345f35c90abfd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112807464096a7814c58ad1345f35c90abfd 2023-04-03 02:54:31,610 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/03e557640cd44021b84d41bb01966f36 2023-04-03 02:54:31,618 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/03e557640cd44021b84d41bb01966f36 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36 2023-04-03 02:54:31,624 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:54:31,624 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 48ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:31,625 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:31,625 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:31,625 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:31,625 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:31,625 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,625 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:31,625 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:31,626 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,627 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,627 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:31,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:31,628 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@391ed55c[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:31,628 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:31,628 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,631 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:31,631 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,631 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:31,631 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:31,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:31,637 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:31,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:31,642 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:31,642 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:31,642 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:31,642 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,642 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:31,645 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,645 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,645 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:31,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:31,645 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:31,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,647 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:31,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:31,658 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,659 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:31,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:31,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:31,659 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:31,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:31,660 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:31,660 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:31,660 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:31,760 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:31,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:31,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:31,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:31,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,769 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:31,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:31,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:31,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:31,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,770 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,771 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:31,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:31,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:31,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:31,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:31,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:31,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:31,774 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:31,774 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:31,774 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:31,774 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:31,774 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:32,199 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201fcdd861b00af4753a8a826215250c69e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201fcdd861b00af4753a8a826215250c69e 2023-04-03 02:54:32,606 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e5e3b9793700454dbe62401a8b63a5b0 2023-04-03 02:54:32,612 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e5e3b9793700454dbe62401a8b63a5b0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0 2023-04-03 02:54:32,617 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0, entries=1, sequenceid=13, filesize=5.0 K 2023-04-03 02:54:32,618 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 844ms, sequenceid=13, compaction requested=true 2023-04-03 02:54:32,618 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:32,618 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:32,618 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:32,618 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:32,618 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,618 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:32,618 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:32,620 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,620 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:32,620 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,620 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:32,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:32,622 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5dc16bf5[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:32,622 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:32,622 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,623 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,623 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:32,623 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:32,623 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:32,624 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,624 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:32,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:32,626 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:32,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:32,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:32,629 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:32,629 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:32,629 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:32,629 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,629 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:32,631 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,631 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:32,631 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:32,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:32,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:32,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,636 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:32,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,637 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:32,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:32,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,639 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:32,639 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:32,640 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:32,640 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:32,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:32,640 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:32,640 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:32,740 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:32,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:32,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:32,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:32,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,748 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:32,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:32,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:32,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:32,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,748 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,749 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:32,749 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:32,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:32,750 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:32,750 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:32,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:32,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,751 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,751 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:32,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:32,751 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:32,751 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:32,751 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:32,751 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:32,752 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:32,752 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:32,752 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:32,752 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:33,178 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d01974b2501a46d7a44a6eb5bba53e26 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d01974b2501a46d7a44a6eb5bba53e26 2023-04-03 02:54:33,586 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/0b54f24925cf448ab704ae6ebb4338ba 2023-04-03 02:54:33,591 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/0b54f24925cf448ab704ae6ebb4338ba as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba 2023-04-03 02:54:33,596 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba, entries=1, sequenceid=17, filesize=5.0 K 2023-04-03 02:54:33,597 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 845ms, sequenceid=17, compaction requested=true 2023-04-03 02:54:33,598 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:33,598 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:33,598 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:33,599 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:33,601 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,601 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:33,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:33,602 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,602 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:33,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:33,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:33,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:33,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:33,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19e39f8[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:33,603 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:33,603 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,604 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,605 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:33,605 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:33,605 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:33,606 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:33,606 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,606 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:33,606 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:33,606 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:33,606 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:33,606 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:33,606 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:33,607 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:33,607 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,607 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:33,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:33,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:33,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:33,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:33,610 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:33,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,611 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:33,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,614 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:33,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:33,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:33,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:33,614 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,615 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:33,615 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:33,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:33,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:33,615 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:33,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:33,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:33,715 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:33,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:33,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:33,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:33,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,722 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:33,722 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:33,722 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:33,722 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:33,723 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,723 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,724 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:33,724 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:33,724 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:33,725 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:33,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:33,726 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:33,726 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:33,726 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:33,726 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:34,142 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122889789041bbb4488c88aa1ebad9f9ddb5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122889789041bbb4488c88aa1ebad9f9ddb5 2023-04-03 02:54:34,558 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/af321fbe96474089be62e47fcc1df54b 2023-04-03 02:54:34,570 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/af321fbe96474089be62e47fcc1df54b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b 2023-04-03 02:54:34,578 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b, entries=1, sequenceid=21, filesize=5.0 K 2023-04-03 02:54:34,578 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 852ms, sequenceid=21, compaction requested=true 2023-04-03 02:54:34,579 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:34,579 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:34,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:34,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:34,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:34,579 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:34,583 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,583 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,583 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:34,584 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:34,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:34,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:34,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:34,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:34,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56c23cd5[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:34,586 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:34,586 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,592 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,592 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:34,592 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:34,592 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:34,592 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,592 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:34,594 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:34,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:34,594 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:34,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:34,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:34,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:34,597 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:34,598 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:34,598 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:34,598 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,598 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:34,600 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,600 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:34,600 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:34,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:34,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:34,603 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:34,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,604 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:34,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:34,609 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:34,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:34,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:34,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,609 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,610 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:34,610 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:34,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:34,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:34,710 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:34,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:34,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:34,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:34,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,719 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:34,719 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:34,719 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:34,719 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:34,720 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,720 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,722 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:34,722 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:34,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:34,723 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:34,723 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:34,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:34,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,728 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:34,728 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:34,728 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:34,732 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:34,732 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:34,733 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:34,733 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:34,733 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:34,760 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231047b3f4a90144a03a613907b2735a972 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231047b3f4a90144a03a613907b2735a972 2023-04-03 02:54:35,170 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/386bdb0e9b7046cda3a4e6e99d47e08f 2023-04-03 02:54:35,176 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/386bdb0e9b7046cda3a4e6e99d47e08f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f 2023-04-03 02:54:35,181 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f, entries=1, sequenceid=25, filesize=5.0 K 2023-04-03 02:54:35,182 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 449ms, sequenceid=25, compaction requested=true 2023-04-03 02:54:35,182 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:35,182 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:35,183 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:35,184 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,184 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:35,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:35,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1939a764[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:35,186 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:35,186 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,187 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,187 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:35,188 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:35,188 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:35,188 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,188 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:35,189 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:35,189 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:35,189 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,189 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:35,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:35,192 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:35,192 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:35,192 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:35,192 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,192 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:35,195 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,195 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:35,195 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,195 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:35,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,197 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:35,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:35,201 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:35,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:35,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:35,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:35,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:35,204 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:35,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:35,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:35,204 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:35,204 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:35,304 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:35,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:35,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:35,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:35,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,315 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:35,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:35,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:35,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:35,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,316 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,317 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:35,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:35,317 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:35,317 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:35,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:35,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:35,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:35,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:35,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:35,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:35,321 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:35,321 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:35,321 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:35,321 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:35,322 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:35,746 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101aa2f9d1fa299459dbe4d457875ca5026 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa2f9d1fa299459dbe4d457875ca5026 2023-04-03 02:54:35,849 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:54:36,151 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e3f4bbb48e3343e48f22517ad5ac42c7 2023-04-03 02:54:36,156 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e3f4bbb48e3343e48f22517ad5ac42c7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7 2023-04-03 02:54:36,160 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7, entries=1, sequenceid=29, filesize=5.0 K 2023-04-03 02:54:36,161 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 839ms, sequenceid=29, compaction requested=true 2023-04-03 02:54:36,161 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:36,161 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:36,161 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:36,161 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:36,161 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,161 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:36,161 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:36,163 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,163 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:36,163 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,163 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:36,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:36,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f71e975[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:36,165 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:36,165 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,166 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:36,166 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,166 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:36,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:36,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:36,168 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,168 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:36,168 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:36,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:36,168 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:36,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:36,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:36,170 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:36,171 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:36,171 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:36,171 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,171 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:36,172 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,172 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:36,172 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:36,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:36,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:36,172 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:36,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,174 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:36,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:36,178 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,178 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:36,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:36,178 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:36,179 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:36,179 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:36,179 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:36,279 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:36,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:36,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:36,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:36,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,286 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:36,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:36,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:36,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:36,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,286 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,287 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,287 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:36,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:36,288 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:36,288 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,289 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:36,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:36,289 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:36,289 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:36,289 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:36,289 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:36,289 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:36,289 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:36,289 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:36,289 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:36,703 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030c703fb61aaa480f91a1fc6c38ebf308 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030c703fb61aaa480f91a1fc6c38ebf308 2023-04-03 02:54:37,109 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/ea3d15e84efd473b9cc8046e64a2fdfd 2023-04-03 02:54:37,113 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/ea3d15e84efd473b9cc8046e64a2fdfd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd 2023-04-03 02:54:37,117 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd, entries=1, sequenceid=33, filesize=5.0 K 2023-04-03 02:54:37,117 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 828ms, sequenceid=33, compaction requested=true 2023-04-03 02:54:37,118 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:37,118 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:37,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:37,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:37,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:37,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,119 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,120 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:37,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:37,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f50489a[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:37,121 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:37,121 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,123 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:37,123 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,123 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:37,124 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:37,124 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:37,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,124 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:37,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:37,127 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:37,127 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:37,127 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:37,127 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,127 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:37,129 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:37,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:37,129 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,130 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:37,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:37,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:37,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,133 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,133 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,133 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:37,133 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:37,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:37,233 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:37,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:37,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:37,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:37,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,239 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:37,239 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:37,239 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:37,239 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:37,240 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,240 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,241 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,241 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:37,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:37,242 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:37,242 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,242 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:37,243 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:37,243 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:37,243 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:37,243 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:37,243 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:37,654 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f13f44de123242afa7bf3eeff716f1c0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907f13f44de123242afa7bf3eeff716f1c0 2023-04-03 02:54:38,060 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/7e1a5de2a589408baf96499921bbc583 2023-04-03 02:54:38,064 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/7e1a5de2a589408baf96499921bbc583 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583 2023-04-03 02:54:38,068 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583, entries=1, sequenceid=37, filesize=5.0 K 2023-04-03 02:54:38,068 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 825ms, sequenceid=37, compaction requested=true 2023-04-03 02:54:38,069 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:38,069 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:38,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:38,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:38,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:38,069 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:38,070 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:38,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:38,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,071 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:38,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:38,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78b3ff31[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:38,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:38,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,073 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:38,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:38,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:38,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:38,074 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:38,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:38,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:38,076 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:38,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:38,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:38,076 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,077 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:38,079 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,079 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:38,079 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:38,080 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:38,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:38,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:38,083 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:38,083 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:38,184 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:38,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:38,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:38,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:38,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,190 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:38,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:38,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-03 02:54:38,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-03 02:54:38,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,190 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,194 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:38,194 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:38,194 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:38,194 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-03 02:54:38,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-03 02:54:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-03 02:54:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:38,195 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:38,195 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:38,195 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. started... 2023-04-03 02:54:38,195 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 90d3745b2d090a6eb1d4604d211e6a26 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:38,607 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b4c99c96cc1c4693860720a7ed84c845 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120b4c99c96cc1c4693860720a7ed84c845 2023-04-03 02:54:39,012 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e6fabb2e709e4421ab7b233af59f6004 2023-04-03 02:54:39,017 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/e6fabb2e709e4421ab7b233af59f6004 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004 2023-04-03 02:54:39,020 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004, entries=1, sequenceid=41, filesize=5.0 K 2023-04-03 02:54:39,021 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 90d3745b2d090a6eb1d4604d211e6a26 in 826ms, sequenceid=41, compaction requested=true 2023-04-03 02:54:39,021 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:39,021 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-03 02:54:39,021 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-03 02:54:39,023 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:39,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-03 02:54:39,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ce565c4[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:39,024 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-03 02:54:39,025 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,025 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-03 02:54:39,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-03 02:54:39,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:39,026 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,026 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:39,028 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-03 02:54:39,028 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,028 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:39,028 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-03 02:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:39,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-03 02:54:39,030 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-03 02:54:39,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:39,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:39,030 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,030 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:39,031 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,031 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:39,031 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:39,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:39,032 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,033 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:39,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:39,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:39,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:39,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:39,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:39,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,036 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:39,136 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:39,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:39,138 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:39,139 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:39,140 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:39,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:39,140 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491079140 2023-04-03 02:54:39,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-03 02:54:39,142 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:39,143 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x1546f10a to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:39,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@454779ad, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:39,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:39,986 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032464b4a3b21949729247e3a251aba790 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601032464b4a3b21949729247e3a251aba790 2023-04-03 02:54:39,988 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:40,000 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:40,002 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50126, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:40,008 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:40,009 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39924, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:40,017 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/793e7ba7bd854418b3a442119e39c544 first=Optional[r3] last=Optional[r6] 2023-04-03 02:54:40,018 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/793e7ba7bd854418b3a442119e39c544}] 2023-04-03 02:54:40,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/793e7ba7bd854418b3a442119e39c544 for inclusion in 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:40,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-03 02:54:40,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:40,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:40,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/793e7ba7bd854418b3a442119e39c544 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vqtkjceu66rnr3qjpch56fa7kb8chlpd5srplndh750m02et3ipgpalstbil4fjl/family1/793e7ba7bd854418b3a442119e39c544 2023-04-03 02:54:40,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vqtkjceu66rnr3qjpch56fa7kb8chlpd5srplndh750m02et3ipgpalstbil4fjl/family1/793e7ba7bd854418b3a442119e39c544 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_ 2023-04-03 02:54:40,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vqtkjceu66rnr3qjpch56fa7kb8chlpd5srplndh750m02et3ipgpalstbil4fjl/family1/793e7ba7bd854418b3a442119e39c544 into 90d3745b2d090a6eb1d4604d211e6a26/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_ - updating store file list. 2023-04-03 02:54:40,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_ into 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:40,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vqtkjceu66rnr3qjpch56fa7kb8chlpd5srplndh750m02et3ipgpalstbil4fjl/family1/793e7ba7bd854418b3a442119e39c544 into 90d3745b2d090a6eb1d4604d211e6a26/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_) 2023-04-03 02:54:40,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__vqtkjceu66rnr3qjpch56fa7kb8chlpd5srplndh750m02et3ipgpalstbil4fjl/family1/793e7ba7bd854418b3a442119e39c544 2023-04-03 02:54:40,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:54:40,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:54:40,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 90d3745b2d090a6eb1d4604d211e6a26/family1 is initiating minor compaction 2023-04-03 02:54:40,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7c50af01; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:40,035 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 90d3745b2d090a6eb1d4604d211e6a26/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:40,035 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp, totalSize=50.4 K 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 03e557640cd44021b84d41bb01966f36, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e5e3b9793700454dbe62401a8b63a5b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 0b54f24925cf448ab704ae6ebb4338ba, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting af321fbe96474089be62e47fcc1df54b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 386bdb0e9b7046cda3a4e6e99d47e08f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e3f4bbb48e3343e48f22517ad5ac42c7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting ea3d15e84efd473b9cc8046e64a2fdfd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:54:40,035 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7e1a5de2a589408baf96499921bbc583, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:54:40,036 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e6fabb2e709e4421ab7b233af59f6004, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:54:40,036 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e57acd763a81459dbfb74b38b7775b6f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:54:40,036 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:40,037 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:40,039 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122889789041bbb4488c88aa1ebad9f9ddb5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015122889789041bbb4488c88aa1ebad9f9ddb5 2023-04-03 02:54:40,040 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231047b3f4a90144a03a613907b2735a972 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231047b3f4a90144a03a613907b2735a972 2023-04-03 02:54:40,041 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa2f9d1fa299459dbe4d457875ca5026 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa2f9d1fa299459dbe4d457875ca5026 2023-04-03 02:54:40,042 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030c703fb61aaa480f91a1fc6c38ebf308 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601030c703fb61aaa480f91a1fc6c38ebf308 2023-04-03 02:54:40,042 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-04-03 02:54:40,042 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,067 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 90d3745b2d090a6eb1d4604d211e6a26#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:40,487 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/4b6e90d4f9474840b8f70e4630767c6d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/4b6e90d4f9474840b8f70e4630767c6d 2023-04-03 02:54:40,488 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120575a2fd25ef214044b0a495dea3a7301e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120575a2fd25ef214044b0a495dea3a7301e 2023-04-03 02:54:40,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:40,491 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 90d3745b2d090a6eb1d4604d211e6a26/family1 of 90d3745b2d090a6eb1d4604d211e6a26 into 4b6e90d4f9474840b8f70e4630767c6d(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:40,491 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:40,492 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1680490480034; duration=0sec 2023-04-03 02:54:40,492 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:40,514 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/12de3ea918a04f28ab07e53f804da8b6 first=Optional[r00] last=Optional[r2] 2023-04-03 02:54:40,514 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/12de3ea918a04f28ab07e53f804da8b6}] 2023-04-03 02:54:40,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/12de3ea918a04f28ab07e53f804da8b6 for inclusion in 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:40,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-03 02:54:40,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:40,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:40,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/12de3ea918a04f28ab07e53f804da8b6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__bodfn2guu9750b5bjealvopkj5st48j807jbjpvkt5vesevn2d32d3p5bq8v9mj3/family1/12de3ea918a04f28ab07e53f804da8b6 2023-04-03 02:54:40,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__bodfn2guu9750b5bjealvopkj5st48j807jbjpvkt5vesevn2d32d3p5bq8v9mj3/family1/12de3ea918a04f28ab07e53f804da8b6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_ 2023-04-03 02:54:40,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__bodfn2guu9750b5bjealvopkj5st48j807jbjpvkt5vesevn2d32d3p5bq8v9mj3/family1/12de3ea918a04f28ab07e53f804da8b6 into 90d3745b2d090a6eb1d4604d211e6a26/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_ - updating store file list. 2023-04-03 02:54:40,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_ into 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:40,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__bodfn2guu9750b5bjealvopkj5st48j807jbjpvkt5vesevn2d32d3p5bq8v9mj3/family1/12de3ea918a04f28ab07e53f804da8b6 into 90d3745b2d090a6eb1d4604d211e6a26/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_) 2023-04-03 02:54:40,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__bodfn2guu9750b5bjealvopkj5st48j807jbjpvkt5vesevn2d32d3p5bq8v9mj3/family1/12de3ea918a04f28ab07e53f804da8b6 2023-04-03 02:54:40,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:54:40,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:54:40,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 90d3745b2d090a6eb1d4604d211e6a26/family1 is initiating minor compaction (all files) 2023-04-03 02:54:40,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@79df0532; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:40,526 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 90d3745b2d090a6eb1d4604d211e6a26/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:40,526 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/4b6e90d4f9474840b8f70e4630767c6d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp, totalSize=390.1 K 2023-04-03 02:54:40,527 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting a9bb1575299d4d3d86804e49667edc1c, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:54:40,527 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 4b6e90d4f9474840b8f70e4630767c6d, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:54:40,527 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878844911 2023-04-03 02:54:40,528 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:40,529 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:40,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130bab02b7c34c044e6b70e4a310fdf29eb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130bab02b7c34c044e6b70e4a310fdf29eb 2023-04-03 02:54:40,531 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201fcdd861b00af4753a8a826215250c69e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201fcdd861b00af4753a8a826215250c69e 2023-04-03 02:54:40,533 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d01974b2501a46d7a44a6eb5bba53e26 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d01974b2501a46d7a44a6eb5bba53e26 2023-04-03 02:54:40,533 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-04-03 02:54:40,533 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-03 02:54:40,533 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:40,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:40,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:40,535 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:40,536 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.3970 sec 2023-04-03 02:54:40,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_] to archive 2023-04-03 02:54:40,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:40,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/03e557640cd44021b84d41bb01966f36 2023-04-03 02:54:40,544 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e5e3b9793700454dbe62401a8b63a5b0 2023-04-03 02:54:40,545 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 90d3745b2d090a6eb1d4604d211e6a26#family1#compaction#61 average throughput is 18.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:40,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/0b54f24925cf448ab704ae6ebb4338ba 2023-04-03 02:54:40,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/af321fbe96474089be62e47fcc1df54b 2023-04-03 02:54:40,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/386bdb0e9b7046cda3a4e6e99d47e08f 2023-04-03 02:54:40,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e3f4bbb48e3343e48f22517ad5ac42c7 2023-04-03 02:54:40,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/ea3d15e84efd473b9cc8046e64a2fdfd 2023-04-03 02:54:40,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7e1a5de2a589408baf96499921bbc583 2023-04-03 02:54:40,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e6fabb2e709e4421ab7b233af59f6004 2023-04-03 02:54:40,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/e57acd763a81459dbfb74b38b7775b6f_SeqId_44_ 2023-04-03 02:54:40,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-03 02:54:40,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:41,022 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:41,024 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:41,024 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:41,026 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-04-03 02:54:41,028 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE}] 2023-04-03 02:54:41,029 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE 2023-04-03 02:54:41,029 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:41,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490481029"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490481029"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490481029"}]},"ts":"1680490481029"} 2023-04-03 02:54:41,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:41,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:41,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 90d3745b2d090a6eb1d4604d211e6a26, disabling compactions & flushes 2023-04-03 02:54:41,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:41,774 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/.tmp/family1/7d88fb54f0594faab25d962d4ccdff37 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7d88fb54f0594faab25d962d4ccdff37 2023-04-03 02:54:41,778 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 90d3745b2d090a6eb1d4604d211e6a26/family1 of 90d3745b2d090a6eb1d4604d211e6a26 into 7d88fb54f0594faab25d962d4ccdff37(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:54:41,778 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:41,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:41,778 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1680490480526; duration=1sec 2023-04-03 02:54:41,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:41,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. after waiting 0 ms 2023-04-03 02:54:41,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:41,778 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:41,779 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/4b6e90d4f9474840b8f70e4630767c6d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_] to archive 2023-04-03 02:54:41,779 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:41,781 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/a9bb1575299d4d3d86804e49667edc1c 2023-04-03 02:54:41,782 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/4b6e90d4f9474840b8f70e4630767c6d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/4b6e90d4f9474840b8f70e4630767c6d 2023-04-03 02:54:41,783 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/6aeae00e7c5b4c93a88811a535e8aec9_SeqId_47_ 2023-04-03 02:54:41,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-03 02:54:41,787 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:41,787 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:41,788 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 90d3745b2d090a6eb1d4604d211e6a26 to self. 2023-04-03 02:54:41,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:41,789 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=CLOSED 2023-04-03 02:54:41,789 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490481789"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490481789"}]},"ts":"1680490481789"} 2023-04-03 02:54:41,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-04-03 02:54:41,792 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719 in 760 msec 2023-04-03 02:54:41,792 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=true 2023-04-03 02:54:41,943 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:41,943 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490481943"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490481943"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490481943"}]},"ts":"1680490481943"} 2023-04-03 02:54:41,944 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:42,036 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-03 02:54:42,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:42,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 90d3745b2d090a6eb1d4604d211e6a26, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,102 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,102 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:42,103 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:42,103 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90d3745b2d090a6eb1d4604d211e6a26 columnFamilyName family1 2023-04-03 02:54:42,107 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7d88fb54f0594faab25d962d4ccdff37 2023-04-03 02:54:42,108 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(310): Store=90d3745b2d090a6eb1d4604d211e6a26/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:42,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:42,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 90d3745b2d090a6eb1d4604d211e6a26; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75493056, jitterRate=0.12493419647216797}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:42,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:42,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., pid=47, masterSystemTime=1680490482096 2023-04-03 02:54:42,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:42,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:42,115 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:42,115 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490482115"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490482115"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490482115"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490482115"}]},"ts":"1680490482115"} 2023-04-03 02:54:42,118 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-04-03 02:54:42,118 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719 in 172 msec 2023-04-03 02:54:42,119 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-04-03 02:54:42,119 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE in 1.0900 sec 2023-04-03 02:54:42,121 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-04-03 02:54:42,121 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0930 sec 2023-04-03 02:54:42,122 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5100 sec 2023-04-03 02:54:42,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-04-03 02:54:42,124 INFO [Listener at localhost/37595] compactions.TestMobCompactor(278): alter status finished 2023-04-03 02:54:42,125 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:42,126 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:42,126 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:42,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:42,127 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491082127 2023-04-03 02:54:42,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:42,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-03 02:54:42,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:42,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x535252e4 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:42,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2da6d332, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:42,133 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:42,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:42,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:42,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:42,134 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:42,960 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e8440d3b7aa84748b9a9cf71d37af3e1 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128e8440d3b7aa84748b9a9cf71d37af3e1 2023-04-03 02:54:42,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:42,973 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:42,974 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50128, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:42,980 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:42,982 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39938, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:42,990 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d5c97d5ed24a43fa9bb7661fc8f91cba first=Optional[r05] last=Optional[r8] 2023-04-03 02:54:42,991 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=51 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d5c97d5ed24a43fa9bb7661fc8f91cba}] 2023-04-03 02:54:42,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d5c97d5ed24a43fa9bb7661fc8f91cba for inclusion in 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:42,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-03 02:54:42,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:42,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:42,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/d5c97d5ed24a43fa9bb7661fc8f91cba to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5d05asercnt06a4cqk1kb37igbsjfk90eposn8uq809a6edmicqvgrp28sdg5hlt/family1/d5c97d5ed24a43fa9bb7661fc8f91cba 2023-04-03 02:54:43,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5d05asercnt06a4cqk1kb37igbsjfk90eposn8uq809a6edmicqvgrp28sdg5hlt/family1/d5c97d5ed24a43fa9bb7661fc8f91cba as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/cddf50626123461189ddbbf6e4829b3d_SeqId_53_ 2023-04-03 02:54:43,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5d05asercnt06a4cqk1kb37igbsjfk90eposn8uq809a6edmicqvgrp28sdg5hlt/family1/d5c97d5ed24a43fa9bb7661fc8f91cba into 90d3745b2d090a6eb1d4604d211e6a26/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/cddf50626123461189ddbbf6e4829b3d_SeqId_53_ - updating store file list. 2023-04-03 02:54:43,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/cddf50626123461189ddbbf6e4829b3d_SeqId_53_ into 90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:43,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5d05asercnt06a4cqk1kb37igbsjfk90eposn8uq809a6edmicqvgrp28sdg5hlt/family1/d5c97d5ed24a43fa9bb7661fc8f91cba into 90d3745b2d090a6eb1d4604d211e6a26/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/cddf50626123461189ddbbf6e4829b3d_SeqId_53_) 2023-04-03 02:54:43,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5d05asercnt06a4cqk1kb37igbsjfk90eposn8uq809a6edmicqvgrp28sdg5hlt/family1/d5c97d5ed24a43fa9bb7661fc8f91cba 2023-04-03 02:54:43,007 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:43,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:43,009 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120b4c99c96cc1c4693860720a7ed84c845 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120b4c99c96cc1c4693860720a7ed84c845 2023-04-03 02:54:43,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112807464096a7814c58ad1345f35c90abfd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112807464096a7814c58ad1345f35c90abfd 2023-04-03 02:54:43,010 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-04-03 02:54:43,010 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-03 02:54:43,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,011 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:43,011 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:43,012 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:43,013 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 887 msec 2023-04-03 02:54:43,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:54:43,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:43,474 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-04-03 02:54:43,475 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:43,476 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-04-03 02:54:43,477 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-04-03 02:54:43,481 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE}] 2023-04-03 02:54:43,482 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE 2023-04-03 02:54:43,482 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:43,483 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490483482"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490483482"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490483482"}]},"ts":"1680490483482"} 2023-04-03 02:54:43,484 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:43,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 90d3745b2d090a6eb1d4604d211e6a26, disabling compactions & flushes 2023-04-03 02:54:43,636 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. after waiting 0 ms 2023-04-03 02:54:43,636 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-04-03 02:54:43,642 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:43,642 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 90d3745b2d090a6eb1d4604d211e6a26 to self. 2023-04-03 02:54:43,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,643 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=CLOSED 2023-04-03 02:54:43,643 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490483643"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490483643"}]},"ts":"1680490483643"} 2023-04-03 02:54:43,645 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-04-03 02:54:43,646 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719 in 160 msec 2023-04-03 02:54:43,646 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=true 2023-04-03 02:54:43,796 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:43,797 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490483796"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490483796"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490483796"}]},"ts":"1680490483796"} 2023-04-03 02:54:43,798 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:43,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 90d3745b2d090a6eb1d4604d211e6a26, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:43,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:43,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,954 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,955 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:43,955 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1 2023-04-03 02:54:43,956 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 90d3745b2d090a6eb1d4604d211e6a26 columnFamilyName family1 2023-04-03 02:54:43,960 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/7d88fb54f0594faab25d962d4ccdff37 2023-04-03 02:54:43,963 DEBUG [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/family1/cddf50626123461189ddbbf6e4829b3d_SeqId_53_ 2023-04-03 02:54:43,963 INFO [StoreOpener-90d3745b2d090a6eb1d4604d211e6a26-1] regionserver.HStore(310): Store=90d3745b2d090a6eb1d4604d211e6a26/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:43,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:54:43,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 90d3745b2d090a6eb1d4604d211e6a26; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73300817, jitterRate=0.09226728975772858}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:43,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:54:43,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26., pid=53, masterSystemTime=1680490483949 2023-04-03 02:54:43,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:54:43,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=90d3745b2d090a6eb1d4604d211e6a26, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:43,971 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1680490483971"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490483971"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490483971"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490483971"}]},"ts":"1680490483971"} 2023-04-03 02:54:43,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-04-03 02:54:43,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 90d3745b2d090a6eb1d4604d211e6a26, server=jenkins-hbase4.apache.org,45927,1680490420719 in 175 msec 2023-04-03 02:54:43,975 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-04-03 02:54:43,975 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=90d3745b2d090a6eb1d4604d211e6a26, REOPEN/MOVE in 493 msec 2023-04-03 02:54:43,977 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-04-03 02:54:43,977 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 499 msec 2023-04-03 02:54:43,978 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 912 msec 2023-04-03 02:54:43,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-03 02:54:43,980 INFO [Listener at localhost/37595] compactions.TestMobCompactor(278): alter status finished 2023-04-03 02:54:43,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-03 02:54:43,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:43,983 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:43,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:43,983 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491083983 2023-04-03 02:54:43,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:43,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-03 02:54:43,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:43,985 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x56e89800 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:43,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d4cfeac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-03 02:54:43,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-03 02:54:43,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:43,991 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-03 02:54:43,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-04-03 02:54:44,038 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=252 (was 250) - Thread LEAK? -, OpenFileDescriptor=631 (was 603) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=430 (was 484), ProcessCount=170 (was 170), AvailableMemoryMB=3582 (was 3590) 2023-04-03 02:54:44,048 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=252, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=430, ProcessCount=170, AvailableMemoryMB=3582 2023-04-03 02:54:44,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-03 02:54:44,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:44,052 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:44,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-04-03 02:54:44,052 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:44,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:44,054 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:44,054 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 empty. 2023-04-03 02:54:44,054 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:44,054 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-04-03 02:54:44,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:44,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:44,461 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:44,462 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1af443f08637505427041327164c21b5, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:44,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 1af443f08637505427041327164c21b5, disabling compactions & flushes 2023-04-03 02:54:44,868 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. after waiting 0 ms 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:44,868 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:44,868 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:44,870 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:44,871 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490484871"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490484871"}]},"ts":"1680490484871"} 2023-04-03 02:54:44,872 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:54:44,873 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:44,873 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490484873"}]},"ts":"1680490484873"} 2023-04-03 02:54:44,874 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-04-03 02:54:44,877 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, ASSIGN}] 2023-04-03 02:54:44,878 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, ASSIGN 2023-04-03 02:54:44,878 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:45,029 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:45,030 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490485029"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490485029"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490485029"}]},"ts":"1680490485029"} 2023-04-03 02:54:45,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:45,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:45,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:45,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1af443f08637505427041327164c21b5, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:45,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1af443f08637505427041327164c21b5 2023-04-03 02:54:45,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:45,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1af443f08637505427041327164c21b5 2023-04-03 02:54:45,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1af443f08637505427041327164c21b5 2023-04-03 02:54:45,187 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1af443f08637505427041327164c21b5 2023-04-03 02:54:45,189 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:45,189 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:45,189 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af443f08637505427041327164c21b5 columnFamilyName family1 2023-04-03 02:54:45,190 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(310): Store=1af443f08637505427041327164c21b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:45,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:45,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:45,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1af443f08637505427041327164c21b5 2023-04-03 02:54:45,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:45,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1af443f08637505427041327164c21b5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63332302, jitterRate=-0.05627515912055969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:45,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:45,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., pid=57, masterSystemTime=1680490485183 2023-04-03 02:54:45,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:45,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:45,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:45,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490485199"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490485199"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490485199"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490485199"}]},"ts":"1680490485199"} 2023-04-03 02:54:45,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-04-03 02:54:45,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719 in 169 msec 2023-04-03 02:54:45,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-04-03 02:54:45,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, ASSIGN in 324 msec 2023-04-03 02:54:45,203 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:45,203 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490485203"}]},"ts":"1680490485203"} 2023-04-03 02:54:45,204 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-04-03 02:54:45,206 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:45,207 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1560 sec 2023-04-03 02:54:46,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-03 02:54:46,157 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-04-03 02:54:46,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:46,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,179 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:46,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:46,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:46,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:46,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,180 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:46,180 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:46,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:46,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:46,180 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:46,180 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:46,181 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:46,181 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:46,184 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:46,184 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:46,184 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:46,184 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:46,185 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:54:46,604 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130fa208c007cd942e3a7effaa24fe76cb3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130fa208c007cd942e3a7effaa24fe76cb3 2023-04-03 02:54:47,008 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/e98a022dddca4093873af976e016f0d6 2023-04-03 02:54:47,013 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/e98a022dddca4093873af976e016f0d6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6 2023-04-03 02:54:47,017 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6, entries=1000, sequenceid=5, filesize=190.5 K 2023-04-03 02:54:47,017 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 832ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:47,018 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:47,018 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:47,018 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,020 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,020 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:47,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c1f0b6c[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:47,021 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:47,021 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,023 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:47,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:47,023 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:47,023 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,023 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:47,024 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:47,024 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,024 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,024 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:47,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:47,026 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:47,026 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:47,026 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:47,026 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,026 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:47,028 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,028 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,028 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,028 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,029 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:47,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:47,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:47,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:47,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:47,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:47,035 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,135 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:47,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:47,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:47,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:47,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,142 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:47,142 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:47,142 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:47,142 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:47,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,143 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,144 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,144 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:47,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:47,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:47,144 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,144 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:47,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:47,146 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:47,146 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:47,146 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:54:47,322 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:54:47,558 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128850d17088d1c4172955d8bd0676a226c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128850d17088d1c4172955d8bd0676a226c 2023-04-03 02:54:47,964 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/a62b9c15c9e64401923e2fb2228e88c2 2023-04-03 02:54:47,969 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/a62b9c15c9e64401923e2fb2228e88c2 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2 2023-04-03 02:54:47,972 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:54:47,973 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 827ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:47,973 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:47,973 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:47,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:47,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:47,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:47,973 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,975 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:47,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:47,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51b68aed[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:47,976 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:47,976 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,977 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,977 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:47,977 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:47,978 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:47,978 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,978 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:47,979 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:47,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,979 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:47,980 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:47,981 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:47,981 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:47,981 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:47,981 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,981 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:47,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:47,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:47,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:47,983 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,984 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:47,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:47,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:47,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:47,986 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:47,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:47,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:47,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,010 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-03 02:54:48,087 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:48,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:48,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:48,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:48,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,093 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:48,093 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:48,093 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:48,093 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:48,093 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,093 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,094 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:48,094 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:48,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:48,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:48,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:48,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:48,096 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:48,096 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:48,096 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:48,096 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:48,508 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120186ddfbafaf0c43538e05b133dba4f32b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120186ddfbafaf0c43538e05b133dba4f32b 2023-04-03 02:54:48,913 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b3325e15cc6e43f785bd8c914c83c814 2023-04-03 02:54:48,917 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b3325e15cc6e43f785bd8c914c83c814 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814 2023-04-03 02:54:48,921 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814, entries=1, sequenceid=13, filesize=5.0 K 2023-04-03 02:54:48,922 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 826ms, sequenceid=13, compaction requested=true 2023-04-03 02:54:48,922 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:48,922 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:48,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:48,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:48,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:48,922 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:48,927 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:48,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,927 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:48,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:48,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:48,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@369bad9d[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:48,928 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:48,928 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,933 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:48,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:48,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:48,934 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,934 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:48,941 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,941 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:48,941 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:48,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:48,942 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:48,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:48,944 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:48,944 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:48,944 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:48,944 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,944 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:48,953 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,953 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:48,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:48,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,953 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:48,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:48,954 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,955 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:48,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,964 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:48,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:48,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:48,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:48,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:48,964 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,965 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,965 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:48,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:48,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:48,965 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:48,965 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:49,065 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:49,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:49,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:49,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:49,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,071 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:49,071 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:49,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:49,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:49,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,072 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,073 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:49,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:49,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,073 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:49,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:49,074 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:49,075 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:49,075 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:49,075 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:49,075 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:49,075 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:49,490 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050a5fe2aa18904d708cb129c9de187338 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050a5fe2aa18904d708cb129c9de187338 2023-04-03 02:54:49,912 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/ecc406fb463d46188d125cf8546d5b8d 2023-04-03 02:54:49,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/ecc406fb463d46188d125cf8546d5b8d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d 2023-04-03 02:54:49,923 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d, entries=1, sequenceid=17, filesize=5.0 K 2023-04-03 02:54:49,924 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 849ms, sequenceid=17, compaction requested=true 2023-04-03 02:54:49,924 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:49,924 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:49,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:49,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:49,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:49,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:49,926 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:49,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:49,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:49,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31d735cf[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:49,928 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:49,928 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,936 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,936 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:49,936 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:49,936 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:49,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:49,937 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,937 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:49,937 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:49,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:49,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:49,938 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:49,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:49,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:49,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:49,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:49,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:49,940 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:49,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:49,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:49,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,941 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:49,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:49,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:49,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:49,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:49,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:49,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:49,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,945 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:49,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,947 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:49,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:49,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-03 02:54:49,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:49,948 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:49,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:49,948 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:49,948 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:49,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:49,948 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:49,948 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,048 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:50,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:50,054 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:50,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:50,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,056 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:50,056 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,056 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:50,056 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:50,056 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,056 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,058 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:50,058 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:50,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:50,059 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,059 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:50,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:50,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,060 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,060 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:50,060 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,060 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:50,060 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,060 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:50,060 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:50,061 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:50,061 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:50,061 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:50,061 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:50,082 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122858cbafe8f72d4fca933eafeb231bdee2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122858cbafe8f72d4fca933eafeb231bdee2 2023-04-03 02:54:50,106 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b2c92c89a4d64e8694de2fcba77c8d9d 2023-04-03 02:54:50,113 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b2c92c89a4d64e8694de2fcba77c8d9d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d 2023-04-03 02:54:50,117 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d, entries=1, sequenceid=21, filesize=5.0 K 2023-04-03 02:54:50,118 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 57ms, sequenceid=21, compaction requested=true 2023-04-03 02:54:50,118 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:50,119 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:50,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:50,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:50,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:50,119 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:50,120 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,120 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,121 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,121 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:50,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:50,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25071cf2[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:50,123 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:50,123 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,124 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,124 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,124 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,124 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:50,125 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:50,125 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:50,126 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:50,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:50,129 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:50,129 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:50,129 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:50,129 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,129 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:50,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:50,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:50,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:50,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,133 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,135 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:50,135 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:50,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:50,135 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:50,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:50,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:50,135 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:50,136 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:50,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:50,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-03 02:54:50,236 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:50,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:50,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:50,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:50,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,245 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:50,245 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,245 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:50,245 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:50,246 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,246 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,247 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:50,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:50,247 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:50,248 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,248 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:50,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:50,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,252 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:50,252 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,252 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:50,252 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,252 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:50,256 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:50,256 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:50,256 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:50,256 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:50,256 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:50,299 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d80be587d94c4c37b52d1453f74ae094 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231d80be587d94c4c37b52d1453f74ae094 2023-04-03 02:54:50,334 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b69024f5871f457788d3bec2fb6bba0c 2023-04-03 02:54:50,343 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/b69024f5871f457788d3bec2fb6bba0c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c 2023-04-03 02:54:50,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c, entries=1, sequenceid=25, filesize=5.0 K 2023-04-03 02:54:50,353 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 97ms, sequenceid=25, compaction requested=true 2023-04-03 02:54:50,353 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:50,354 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:50,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:50,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:50,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:50,356 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:50,358 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:50,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:50,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40830b5d[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:50,360 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:50,360 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,362 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,362 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:50,362 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:50,362 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:50,362 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,362 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:50,363 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,363 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:50,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,363 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:50,363 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:50,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:50,366 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:50,366 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:50,366 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:50,366 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,366 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:50,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,368 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:50,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:50,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:50,369 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:50,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:50,370 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,376 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:50,376 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,376 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,376 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,376 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:50,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:50,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:50,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-03 02:54:50,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:50,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:50,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:50,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:50,378 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:50,378 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:50,478 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:50,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:50,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:50,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:50,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,487 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:50,487 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,488 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:50,488 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:50,488 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,488 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,492 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,492 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:50,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:50,493 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:50,493 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,494 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:50,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:50,510 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:50,520 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:50,520 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:50,520 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:50,520 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:50,520 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:50,944 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601015d383db1edb14f628abbace26080193e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601015d383db1edb14f628abbace26080193e 2023-04-03 02:54:51,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/9a1231e5e4384a3d9391e5f007baa7b1 2023-04-03 02:54:51,363 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/9a1231e5e4384a3d9391e5f007baa7b1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1 2023-04-03 02:54:51,367 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1, entries=1, sequenceid=29, filesize=5.0 K 2023-04-03 02:54:51,368 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 848ms, sequenceid=29, compaction requested=true 2023-04-03 02:54:51,368 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:51,368 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:51,368 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:51,368 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:51,368 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,368 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:51,368 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:51,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,370 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,370 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:51,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b1fa195[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:51,372 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:51,372 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,377 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:51,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:51,377 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:51,377 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,377 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:51,378 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:51,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,378 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:51,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:51,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:51,382 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:51,382 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:51,382 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:51,382 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,382 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:51,385 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,385 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,385 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:51,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:51,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:51,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,386 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,388 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,390 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:51,390 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:51,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:51,390 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:51,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:51,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:51,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:51,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:51,390 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:51,390 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,391 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:51,491 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:51,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:51,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:51,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:51,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,498 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:51,498 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:51,498 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:51,498 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:51,499 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,499 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,500 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:51,500 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:51,500 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,500 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:51,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,501 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:51,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,501 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:51,501 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:51,502 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:51,502 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:51,502 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:51,502 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:51,502 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:51,502 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:51,521 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039eddd5f1e2334f9194ce30f3a53810fd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601039eddd5f1e2334f9194ce30f3a53810fd 2023-04-03 02:54:51,932 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/74cb2517e430470c96be498882ee6962 2023-04-03 02:54:51,939 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/74cb2517e430470c96be498882ee6962 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962 2023-04-03 02:54:51,944 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962, entries=1, sequenceid=33, filesize=5.0 K 2023-04-03 02:54:51,945 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 443ms, sequenceid=33, compaction requested=true 2023-04-03 02:54:51,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:51,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:51,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:51,947 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,947 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,947 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,947 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:51,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:51,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23ee8074[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:51,949 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:51,949 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,951 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:51,952 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,952 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:51,953 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:51,953 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:51,953 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:51,953 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:51,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:51,957 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:51,957 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:51,957 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:51,957 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,957 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:51,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:51,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:51,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,959 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:51,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:51,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:51,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:51,960 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,963 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:51,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:51,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,966 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:51,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:51,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:51,967 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:51,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:51,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:51,967 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:51,967 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:52,067 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:52,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:52,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:52,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,074 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,075 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,077 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:52,077 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:52,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:52,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:52,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:52,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:52,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:52,079 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:52,079 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:52,079 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:52,079 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:52,153 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907bc48d9a6cb0b49849f8f7edd08150e3e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907bc48d9a6cb0b49849f8f7edd08150e3e 2023-04-03 02:54:52,171 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/f07f061e8366457fb961a2f5a9933acf 2023-04-03 02:54:52,177 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/f07f061e8366457fb961a2f5a9933acf as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf 2023-04-03 02:54:52,185 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf, entries=1, sequenceid=37, filesize=5.0 K 2023-04-03 02:54:52,186 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 107ms, sequenceid=37, compaction requested=true 2023-04-03 02:54:52,186 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:52,186 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:52,186 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:52,186 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:52,186 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,187 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:52,187 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:52,188 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,188 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:52,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:52,189 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,189 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:52,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:52,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:52,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e93e877[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:52,191 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:52,191 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,192 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:52,192 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:52,192 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,192 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:52,193 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:52,194 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:52,194 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:52,194 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:52,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:52,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:52,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:52,197 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:52,197 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:52,197 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:52,197 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,197 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:52,200 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,200 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,200 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:52,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:52,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:52,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:52,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:52,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:52,201 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:52,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:52,203 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,206 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,206 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:52,206 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:52,206 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:52,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-03 02:54:52,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:52,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:52,207 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:52,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:52,207 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:52,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:52,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:52,307 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:52,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:52,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:54:52,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-03 02:54:52,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,315 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:52,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:52,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-03 02:54:52,315 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-03 02:54:52,316 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,316 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,317 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:52,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:52,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:52,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,317 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,318 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:52,318 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:54:52,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:54:52,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-03 02:54:52,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:52,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-03 02:54:52,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:54:52,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-03 02:54:52,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:54:52,319 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:54:52,319 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:52,319 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. started... 2023-04-03 02:54:52,319 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1af443f08637505427041327164c21b5 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:54:52,749 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120224ae147374c4674a909b279717047c0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120224ae147374c4674a909b279717047c0 2023-04-03 02:54:53,155 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/55c9ce2de63e4817aba130d3c3040fd4 2023-04-03 02:54:53,160 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/55c9ce2de63e4817aba130d3c3040fd4 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4 2023-04-03 02:54:53,165 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4, entries=1, sequenceid=41, filesize=5.0 K 2023-04-03 02:54:53,165 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af443f08637505427041327164c21b5 in 846ms, sequenceid=41, compaction requested=true 2023-04-03 02:54:53,166 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:53,166 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:53,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:54:53,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:54:53,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-03 02:54:53,166 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-03 02:54:53,167 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,167 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:53,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:53,168 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,168 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:54:53,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:53,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:53,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:53,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-03 02:54:53,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6692c51b[Count = 0] remaining members to acquire global barrier 2023-04-03 02:54:53,170 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-03 02:54:53,170 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,171 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-03 02:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-03 02:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:54:53,172 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,172 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-03 02:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:53,173 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-03 02:54:53,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:53,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:53,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:53,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:53,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:53,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:54:53,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-03 02:54:53,176 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-03 02:54:53,176 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:54:53,176 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:54:53,176 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,176 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:54:53,177 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,177 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,177 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:54:53,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,178 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,179 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,182 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:54:53,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-03 02:54:53,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-03 02:54:53,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-03 02:54:53,182 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:54:53,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:54:53,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:54:53,283 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:54:53,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:54:53,285 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:53,287 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:53,288 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:53,289 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:53,289 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491093289 2023-04-03 02:54:53,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:53,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-03 02:54:53,291 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:53,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x5f4922fc to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:53,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@40faaa97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:53,307 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,388 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ac7f057d9eb9400291c6e75d6d7b516a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ac7f057d9eb9400291c6e75d6d7b516a 2023-04-03 02:54:53,390 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:53,406 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:53,407 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50138, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:53,415 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:53,417 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50852, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:53,427 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/eba8d01b0ab44da2ae095851987d25f2 first=Optional[r3] last=Optional[r6] 2023-04-03 02:54:53,432 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/eba8d01b0ab44da2ae095851987d25f2}] 2023-04-03 02:54:53,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/eba8d01b0ab44da2ae095851987d25f2 for inclusion in 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:53,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-03 02:54:53,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:53,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:53,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/eba8d01b0ab44da2ae095851987d25f2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9orhoelh548ciendkd69u3vs6aeo88nsc2pbto9bjn7todggvg06ke5cdg1vk6qp/family1/eba8d01b0ab44da2ae095851987d25f2 2023-04-03 02:54:53,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9orhoelh548ciendkd69u3vs6aeo88nsc2pbto9bjn7todggvg06ke5cdg1vk6qp/family1/eba8d01b0ab44da2ae095851987d25f2 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_ 2023-04-03 02:54:53,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9orhoelh548ciendkd69u3vs6aeo88nsc2pbto9bjn7todggvg06ke5cdg1vk6qp/family1/eba8d01b0ab44da2ae095851987d25f2 into 1af443f08637505427041327164c21b5/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_ - updating store file list. 2023-04-03 02:54:53,457 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_ into 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:53,457 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9orhoelh548ciendkd69u3vs6aeo88nsc2pbto9bjn7todggvg06ke5cdg1vk6qp/family1/eba8d01b0ab44da2ae095851987d25f2 into 1af443f08637505427041327164c21b5/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_) 2023-04-03 02:54:53,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9orhoelh548ciendkd69u3vs6aeo88nsc2pbto9bjn7todggvg06ke5cdg1vk6qp/family1/eba8d01b0ab44da2ae095851987d25f2 2023-04-03 02:54:53,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:54:53,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:54:53,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 1af443f08637505427041327164c21b5/family1 is initiating minor compaction 2023-04-03 02:54:53,462 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1af443f08637505427041327164c21b5/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:53,463 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp, totalSize=50.3 K 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting a62b9c15c9e64401923e2fb2228e88c2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting b3325e15cc6e43f785bd8c914c83c814, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting ecc406fb463d46188d125cf8546d5b8d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting b2c92c89a4d64e8694de2fcba77c8d9d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting b69024f5871f457788d3bec2fb6bba0c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 9a1231e5e4384a3d9391e5f007baa7b1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 74cb2517e430470c96be498882ee6962, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f07f061e8366457fb961a2f5a9933acf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 55c9ce2de63e4817aba130d3c3040fd4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:54:53,463 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 76bda31089e54c9498a515e44674aee3_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:54:53,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@311e6d86; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:53,467 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:53,468 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:53,471 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122858cbafe8f72d4fca933eafeb231bdee2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122858cbafe8f72d4fca933eafeb231bdee2 2023-04-03 02:54:53,472 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231d80be587d94c4c37b52d1453f74ae094 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231d80be587d94c4c37b52d1453f74ae094 2023-04-03 02:54:53,474 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601015d383db1edb14f628abbace26080193e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601015d383db1edb14f628abbace26080193e 2023-04-03 02:54:53,475 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601039eddd5f1e2334f9194ce30f3a53810fd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601039eddd5f1e2334f9194ce30f3a53810fd 2023-04-03 02:54:53,475 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-04-03 02:54:53,475 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-03 02:54:53,476 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,476 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,476 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,476 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,477 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,477 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,477 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,477 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:53,524 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1af443f08637505427041327164c21b5#family1#compaction#72 average throughput is 0.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:54,359 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/bf80d7bacefe40d0ba5d7ada265bab4c as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/bf80d7bacefe40d0ba5d7ada265bab4c 2023-04-03 02:54:54,360 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059341010ad0c240c2b694781c1edb7fdb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512059341010ad0c240c2b694781c1edb7fdb 2023-04-03 02:54:54,361 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:54,366 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1af443f08637505427041327164c21b5/family1 of 1af443f08637505427041327164c21b5 into bf80d7bacefe40d0ba5d7ada265bab4c(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:54,366 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:54,366 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1680490493462; duration=0sec 2023-04-03 02:54:54,366 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:54,397 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ccbed36763574aa69df1f8265bee51ac first=Optional[r00] last=Optional[r2] 2023-04-03 02:54:54,399 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ccbed36763574aa69df1f8265bee51ac}] 2023-04-03 02:54:54,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ccbed36763574aa69df1f8265bee51ac for inclusion in 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:54,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-03 02:54:54,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:54,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:54,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ccbed36763574aa69df1f8265bee51ac to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__48vmhgctnmnj8airu8ac6lq8cmkqt3993jhl675orb10ks88tek9sjdj1rd6enhn/family1/ccbed36763574aa69df1f8265bee51ac 2023-04-03 02:54:54,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__48vmhgctnmnj8airu8ac6lq8cmkqt3993jhl675orb10ks88tek9sjdj1rd6enhn/family1/ccbed36763574aa69df1f8265bee51ac as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_ 2023-04-03 02:54:54,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__48vmhgctnmnj8airu8ac6lq8cmkqt3993jhl675orb10ks88tek9sjdj1rd6enhn/family1/ccbed36763574aa69df1f8265bee51ac into 1af443f08637505427041327164c21b5/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_ - updating store file list. 2023-04-03 02:54:54,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_ into 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:54,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__48vmhgctnmnj8airu8ac6lq8cmkqt3993jhl675orb10ks88tek9sjdj1rd6enhn/family1/ccbed36763574aa69df1f8265bee51ac into 1af443f08637505427041327164c21b5/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_) 2023-04-03 02:54:54,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__48vmhgctnmnj8airu8ac6lq8cmkqt3993jhl675orb10ks88tek9sjdj1rd6enhn/family1/ccbed36763574aa69df1f8265bee51ac 2023-04-03 02:54:54,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:54:54,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:54:54,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 1af443f08637505427041327164c21b5/family1 is initiating minor compaction (all files) 2023-04-03 02:54:54,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1eabf49b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:54,416 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1af443f08637505427041327164c21b5/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:54,416 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/bf80d7bacefe40d0ba5d7ada265bab4c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp, totalSize=388.1 K 2023-04-03 02:54:54,416 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e98a022dddca4093873af976e016f0d6, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:54:54,417 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting bf80d7bacefe40d0ba5d7ada265bab4c, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:54:54,417 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878844911 2023-04-03 02:54:54,419 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:54,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:54,422 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130fa208c007cd942e3a7effaa24fe76cb3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130fa208c007cd942e3a7effaa24fe76cb3 2023-04-03 02:54:54,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120186ddfbafaf0c43538e05b133dba4f32b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120186ddfbafaf0c43538e05b133dba4f32b 2023-04-03 02:54:54,425 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050a5fe2aa18904d708cb129c9de187338 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512050a5fe2aa18904d708cb129c9de187338 2023-04-03 02:54:54,425 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-04-03 02:54:54,425 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-03 02:54:54,426 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:54,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:54,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:54,428 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:54,429 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.1420 sec 2023-04-03 02:54:54,437 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1af443f08637505427041327164c21b5#family1#compaction#73 average throughput is 16.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:54,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_] to archive 2023-04-03 02:54:54,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:54,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/a62b9c15c9e64401923e2fb2228e88c2 2023-04-03 02:54:54,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b3325e15cc6e43f785bd8c914c83c814 2023-04-03 02:54:54,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ecc406fb463d46188d125cf8546d5b8d 2023-04-03 02:54:54,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b2c92c89a4d64e8694de2fcba77c8d9d 2023-04-03 02:54:54,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/b69024f5871f457788d3bec2fb6bba0c 2023-04-03 02:54:54,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/9a1231e5e4384a3d9391e5f007baa7b1 2023-04-03 02:54:54,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/74cb2517e430470c96be498882ee6962 2023-04-03 02:54:54,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/f07f061e8366457fb961a2f5a9933acf 2023-04-03 02:54:54,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/55c9ce2de63e4817aba130d3c3040fd4 2023-04-03 02:54:54,464 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/76bda31089e54c9498a515e44674aee3_SeqId_44_ 2023-04-03 02:54:54,474 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-04-03 02:54:54,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-03 02:54:54,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:54,976 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:54,977 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:54,978 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:54,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-04-03 02:54:54,984 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE}] 2023-04-03 02:54:54,986 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE 2023-04-03 02:54:54,987 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:54,987 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490494987"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490494987"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490494987"}]},"ts":"1680490494987"} 2023-04-03 02:54:54,989 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:55,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1af443f08637505427041327164c21b5, disabling compactions & flushes 2023-04-03 02:54:55,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,290 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/.tmp/family1/ad2fd1a153e4418ab04d6310c0042a57 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ad2fd1a153e4418ab04d6310c0042a57 2023-04-03 02:54:55,295 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1af443f08637505427041327164c21b5/family1 of 1af443f08637505427041327164c21b5 into ad2fd1a153e4418ab04d6310c0042a57(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:55,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,295 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:55,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. after waiting 0 ms 2023-04-03 02:54:55,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,295 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1680490494415; duration=0sec 2023-04-03 02:54:55,296 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:55,296 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/bf80d7bacefe40d0ba5d7ada265bab4c, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_] to archive 2023-04-03 02:54:55,296 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:55,298 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/e98a022dddca4093873af976e016f0d6 2023-04-03 02:54:55,299 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/bf80d7bacefe40d0ba5d7ada265bab4c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/bf80d7bacefe40d0ba5d7ada265bab4c 2023-04-03 02:54:55,301 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/05f4fce32fb14e4eabc7d2da1ae95263_SeqId_47_ 2023-04-03 02:54:55,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-03 02:54:55,306 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,306 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:55,306 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1af443f08637505427041327164c21b5 to self. 2023-04-03 02:54:55,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,308 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=CLOSED 2023-04-03 02:54:55,308 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490495308"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490495308"}]},"ts":"1680490495308"} 2023-04-03 02:54:55,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-04-03 02:54:55,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719 in 321 msec 2023-04-03 02:54:55,312 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=true 2023-04-03 02:54:55,462 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:55,462 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490495462"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490495462"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490495462"}]},"ts":"1680490495462"} 2023-04-03 02:54:55,464 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:55,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1af443f08637505427041327164c21b5, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:55,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:55,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,621 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,622 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:55,622 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:55,622 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af443f08637505427041327164c21b5 columnFamilyName family1 2023-04-03 02:54:55,628 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ad2fd1a153e4418ab04d6310c0042a57 2023-04-03 02:54:55,628 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(310): Store=1af443f08637505427041327164c21b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:55,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:55,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:55,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1af443f08637505427041327164c21b5 2023-04-03 02:54:55,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1af443f08637505427041327164c21b5; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67150259, jitterRate=6.168335676193237E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:55,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:55,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., pid=63, masterSystemTime=1680490495615 2023-04-03 02:54:55,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:55,639 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:55,639 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490495639"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490495639"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490495639"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490495639"}]},"ts":"1680490495639"} 2023-04-03 02:54:55,642 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-04-03 02:54:55,642 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719 in 176 msec 2023-04-03 02:54:55,645 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-04-03 02:54:55,645 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE in 658 msec 2023-04-03 02:54:55,647 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-04-03 02:54:55,647 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 665 msec 2023-04-03 02:54:55,649 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1070 sec 2023-04-03 02:54:55,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-04-03 02:54:55,651 INFO [Listener at localhost/37595] compactions.TestMobCompactor(278): alter status finished 2023-04-03 02:54:55,652 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:55,654 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:55,654 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:55,655 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491095655 2023-04-03 02:54:55,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:55,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:55,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-03 02:54:55,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:55,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x112c6f7a to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:55,663 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@24de4d1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:55,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:55,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:55,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:55,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:55,664 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:55,700 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:55,700 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f4922fc to 127.0.0.1:52880 2023-04-03 02:54:55,700 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:55,701 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x56e89800 to 127.0.0.1:52880 2023-04-03 02:54:55,701 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:55,701 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:55,701 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x535252e4 to 127.0.0.1:52880 2023-04-03 02:54:55,701 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:55,701 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:55,701 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1546f10a to 127.0.0.1:52880 2023-04-03 02:54:55,701 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:55,701 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:54:55,701 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e802a7d to 127.0.0.1:52880 2023-04-03 02:54:55,702 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:54:56,121 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c94ad321543f4ad597c998c20929525d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128c94ad321543f4ad597c998c20929525d 2023-04-03 02:54:56,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:56,142 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:56,144 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50148, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:56,153 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:56,155 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50854, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:56,169 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0e589e5bf4a84140809fdf652bbb3446 first=Optional[r05] last=Optional[r8] 2023-04-03 02:54:56,170 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=51 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0e589e5bf4a84140809fdf652bbb3446}] 2023-04-03 02:54:56,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0e589e5bf4a84140809fdf652bbb3446 for inclusion in 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:56,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-03 02:54:56,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:56,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:56,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0e589e5bf4a84140809fdf652bbb3446 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ig167263hpcg7mlm0fb2aa6dpvrmunbkbqd8hm2rhh67v1eap90tqp4k1dk2p6g/family1/0e589e5bf4a84140809fdf652bbb3446 2023-04-03 02:54:56,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ig167263hpcg7mlm0fb2aa6dpvrmunbkbqd8hm2rhh67v1eap90tqp4k1dk2p6g/family1/0e589e5bf4a84140809fdf652bbb3446 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/6f6dd7412c4c472bbe0cb37556748660_SeqId_53_ 2023-04-03 02:54:56,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ig167263hpcg7mlm0fb2aa6dpvrmunbkbqd8hm2rhh67v1eap90tqp4k1dk2p6g/family1/0e589e5bf4a84140809fdf652bbb3446 into 1af443f08637505427041327164c21b5/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/6f6dd7412c4c472bbe0cb37556748660_SeqId_53_ - updating store file list. 2023-04-03 02:54:56,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/6f6dd7412c4c472bbe0cb37556748660_SeqId_53_ into 1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:56,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ig167263hpcg7mlm0fb2aa6dpvrmunbkbqd8hm2rhh67v1eap90tqp4k1dk2p6g/family1/0e589e5bf4a84140809fdf652bbb3446 into 1af443f08637505427041327164c21b5/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/6f6dd7412c4c472bbe0cb37556748660_SeqId_53_) 2023-04-03 02:54:56,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ig167263hpcg7mlm0fb2aa6dpvrmunbkbqd8hm2rhh67v1eap90tqp4k1dk2p6g/family1/0e589e5bf4a84140809fdf652bbb3446 2023-04-03 02:54:56,195 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:56,196 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:56,198 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120224ae147374c4674a909b279717047c0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120224ae147374c4674a909b279717047c0 2023-04-03 02:54:56,200 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128850d17088d1c4172955d8bd0676a226c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128850d17088d1c4172955d8bd0676a226c 2023-04-03 02:54:56,200 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-04-03 02:54:56,200 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-03 02:54:56,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,200 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,201 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:56,201 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:56,202 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:56,204 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 549 msec 2023-04-03 02:54:56,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-04-03 02:54:56,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:56,279 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-04-03 02:54:56,280 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-03 02:54:56,280 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-04-03 02:54:56,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-04-03 02:54:56,285 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE}] 2023-04-03 02:54:56,286 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE 2023-04-03 02:54:56,286 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:56,286 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490496286"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490496286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490496286"}]},"ts":"1680490496286"} 2023-04-03 02:54:56,288 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:56,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1af443f08637505427041327164c21b5, disabling compactions & flushes 2023-04-03 02:54:56,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. after waiting 0 ms 2023-04-03 02:54:56,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,444 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-04-03 02:54:56,445 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:56,445 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1af443f08637505427041327164c21b5 to self. 2023-04-03 02:54:56,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,447 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=CLOSED 2023-04-03 02:54:56,447 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490496447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490496447"}]},"ts":"1680490496447"} 2023-04-03 02:54:56,450 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-04-03 02:54:56,450 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719 in 160 msec 2023-04-03 02:54:56,451 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=true 2023-04-03 02:54:56,601 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:56,601 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490496601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490496601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490496601"}]},"ts":"1680490496601"} 2023-04-03 02:54:56,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:56,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1af443f08637505427041327164c21b5, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:56,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:56,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,760 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,761 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:56,761 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1 2023-04-03 02:54:56,762 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af443f08637505427041327164c21b5 columnFamilyName family1 2023-04-03 02:54:56,768 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/6f6dd7412c4c472bbe0cb37556748660_SeqId_53_ 2023-04-03 02:54:56,771 DEBUG [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/family1/ad2fd1a153e4418ab04d6310c0042a57 2023-04-03 02:54:56,771 INFO [StoreOpener-1af443f08637505427041327164c21b5-1] regionserver.HStore(310): Store=1af443f08637505427041327164c21b5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:56,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:56,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5 2023-04-03 02:54:56,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1af443f08637505427041327164c21b5 2023-04-03 02:54:56,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1af443f08637505427041327164c21b5; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69337175, jitterRate=0.033204421401023865}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:56,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:54:56,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5., pid=69, masterSystemTime=1680490496755 2023-04-03 02:54:56,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:54:56,780 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1af443f08637505427041327164c21b5, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:56,780 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1680490496780"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490496780"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490496780"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490496780"}]},"ts":"1680490496780"} 2023-04-03 02:54:56,782 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-04-03 02:54:56,783 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 1af443f08637505427041327164c21b5, server=jenkins-hbase4.apache.org,45927,1680490420719 in 178 msec 2023-04-03 02:54:56,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-04-03 02:54:56,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1af443f08637505427041327164c21b5, REOPEN/MOVE in 498 msec 2023-04-03 02:54:56,786 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-04-03 02:54:56,786 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 503 msec 2023-04-03 02:54:56,788 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 519 msec 2023-04-03 02:54:56,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-04-03 02:54:56,791 INFO [Listener at localhost/37595] compactions.TestMobCompactor(278): alter status finished 2023-04-03 02:54:56,792 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-03 02:54:56,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:56,794 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:56,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:56,795 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491096795 2023-04-03 02:54:56,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:54:56,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-03 02:54:56,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:56,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x22469053 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:56,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47db674e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-03 02:54:56,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-03 02:54:56,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:56,803 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-03 02:54:56,804 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-04-03 02:54:56,854 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=241 (was 252), OpenFileDescriptor=621 (was 631), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 430), ProcessCount=170 (was 170), AvailableMemoryMB=3514 (was 3582) 2023-04-03 02:54:56,866 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=241, OpenFileDescriptor=621, MaxFileDescriptor=60000, SystemLoadAverage=413, ProcessCount=170, AvailableMemoryMB=3514 2023-04-03 02:54:56,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-03 02:54:56,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-04-03 02:54:56,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:54:56,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-04-03 02:54:56,874 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:54:56,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-03 02:54:56,880 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:56,881 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab empty. 2023-04-03 02:54:56,881 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:56,881 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-04-03 02:54:56,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-03 02:54:57,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-03 02:54:57,306 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-04-03 02:54:57,307 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => eeadc61284a31488b8fb04799304e6ab, NAME => 'testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing eeadc61284a31488b8fb04799304e6ab, disabling compactions & flushes 2023-04-03 02:54:57,314 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. after waiting 0 ms 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,314 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,314 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:57,316 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:54:57,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680490497317"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490497317"}]},"ts":"1680490497317"} 2023-04-03 02:54:57,318 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:54:57,319 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:54:57,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490497319"}]},"ts":"1680490497319"} 2023-04-03 02:54:57,320 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-04-03 02:54:57,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, ASSIGN}] 2023-04-03 02:54:57,324 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, ASSIGN 2023-04-03 02:54:57,325 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:54:57,476 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=eeadc61284a31488b8fb04799304e6ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:57,476 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680490497476"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490497476"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490497476"}]},"ts":"1680490497476"} 2023-04-03 02:54:57,478 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure eeadc61284a31488b8fb04799304e6ab, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:54:57,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-03 02:54:57,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eeadc61284a31488b8fb04799304e6ab, NAME => 'testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:54:57,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-04-03 02:54:57,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-04-03 02:54:57,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:54:57,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,637 INFO [StoreOpener-eeadc61284a31488b8fb04799304e6ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,638 DEBUG [StoreOpener-eeadc61284a31488b8fb04799304e6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1 2023-04-03 02:54:57,638 DEBUG [StoreOpener-eeadc61284a31488b8fb04799304e6ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1 2023-04-03 02:54:57,639 INFO [StoreOpener-eeadc61284a31488b8fb04799304e6ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eeadc61284a31488b8fb04799304e6ab columnFamilyName f1 2023-04-03 02:54:57,641 INFO [StoreOpener-eeadc61284a31488b8fb04799304e6ab-1] regionserver.HStore(310): Store=eeadc61284a31488b8fb04799304e6ab/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:54:57,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:54:57,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:54:57,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eeadc61284a31488b8fb04799304e6ab; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74915572, jitterRate=0.11632901430130005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:54:57,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:57,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab., pid=73, masterSystemTime=1680490497629 2023-04-03 02:54:57,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:57,654 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=eeadc61284a31488b8fb04799304e6ab, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:54:57,655 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680490497654"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490497654"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490497654"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490497654"}]},"ts":"1680490497654"} 2023-04-03 02:54:57,658 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-04-03 02:54:57,658 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure eeadc61284a31488b8fb04799304e6ab, server=jenkins-hbase4.apache.org,45927,1680490420719 in 178 msec 2023-04-03 02:54:57,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-04-03 02:54:57,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, ASSIGN in 335 msec 2023-04-03 02:54:57,660 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:54:57,660 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490497660"}]},"ts":"1680490497660"} 2023-04-03 02:54:57,661 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-04-03 02:54:57,664 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:54:57,666 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 794 msec 2023-04-03 02:54:57,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-03 02:54:57,979 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-04-03 02:54:57,980 DEBUG [Listener at localhost/37595] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-04-03 02:54:57,981 INFO [Listener at localhost/37595] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-04-03 02:54:57,989 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-04-03 02:54:57,989 INFO [Listener at localhost/37595] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-04-03 02:54:57,989 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-04-03 02:54:57,994 INFO [Listener at localhost/37595] regionserver.HRegion(2745): Flushing eeadc61284a31488b8fb04799304e6ab 1/1 column families, dataSize=37 B heapSize=368 B 2023-04-03 02:54:58,412 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304038f11a113eb3045cf897b13b6d2cc252a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304038f11a113eb3045cf897b13b6d2cc252a 2023-04-03 02:54:58,818 INFO [Listener at localhost/37595] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp/f1/a39d316d50c449f983f45a14d7a7244f 2023-04-03 02:54:58,823 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp/f1/a39d316d50c449f983f45a14d7a7244f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f 2023-04-03 02:54:58,827 INFO [Listener at localhost/37595] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f, entries=1, sequenceid=5, filesize=4.9 K 2023-04-03 02:54:58,828 INFO [Listener at localhost/37595] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for eeadc61284a31488b8fb04799304e6ab in 834ms, sequenceid=5, compaction requested=false 2023-04-03 02:54:58,828 DEBUG [Listener at localhost/37595] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-04-03 02:54:58,829 DEBUG [Listener at localhost/37595] regionserver.HRegion(2446): Flush status journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:58,831 INFO [Listener at localhost/37595] regionserver.HRegion(2745): Flushing eeadc61284a31488b8fb04799304e6ab 1/1 column families, dataSize=37 B heapSize=368 B 2023-04-03 02:54:58,843 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403e899672510e04a8b946cae4ff870f57a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230403e899672510e04a8b946cae4ff870f57a 2023-04-03 02:54:58,850 INFO [Listener at localhost/37595] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp/f1/759932c22db74b3eb912465e71581a9b 2023-04-03 02:54:58,854 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp/f1/759932c22db74b3eb912465e71581a9b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b 2023-04-03 02:54:58,858 INFO [Listener at localhost/37595] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b, entries=1, sequenceid=9, filesize=4.9 K 2023-04-03 02:54:58,859 INFO [Listener at localhost/37595] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for eeadc61284a31488b8fb04799304e6ab in 28ms, sequenceid=9, compaction requested=false 2023-04-03 02:54:58,859 DEBUG [Listener at localhost/37595] regionserver.HRegion(2446): Flush status journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:58,860 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-04-03 02:54:58,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-03 02:54:58,862 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-03 02:54:58,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:58,863 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491098863 2023-04-03 02:54:58,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:54:58,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-04-03 02:54:58,864 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-04-03 02:54:58,865 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x06eedba3 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:54:58,869 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1a132d27, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:54:58,869 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230403 for table='testGetAfterCompaction' and column='f1' 2023-04-03 02:54:58,912 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:58,912 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:58,963 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:58,963 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,014 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,014 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,065 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,065 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,116 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,116 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,167 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,167 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,217 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,218 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,269 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,269 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,297 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040351a2588bec53491d9316ef7842a48e75 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023040351a2588bec53491d9316ef7842a48e75 2023-04-03 02:54:59,298 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testGetAfterCompaction' column='f1' 2023-04-03 02:54:59,316 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:54:59,317 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50150, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:54:59,321 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,321 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,325 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:54:59,327 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50860, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:54:59,343 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/f1/3efe2d7518cc476186e03d51278a4fd5 first=Optional[r1] last=Optional[r2] 2023-04-03 02:54:59,345 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{f1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/f1/3efe2d7518cc476186e03d51278a4fd5}] 2023-04-03 02:54:59,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/f1/3efe2d7518cc476186e03d51278a4fd5 for inclusion in eeadc61284a31488b8fb04799304e6ab/f1 2023-04-03 02:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-04-03 02:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:54:59,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:59,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/f1/3efe2d7518cc476186e03d51278a4fd5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testGetAfterCompaction__tg40ijoujjnh927acleaqphcfn5h8h17pbsoo18ejst6n11oavfs8d1214frjir9/f1/3efe2d7518cc476186e03d51278a4fd5 2023-04-03 02:54:59,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testGetAfterCompaction__tg40ijoujjnh927acleaqphcfn5h8h17pbsoo18ejst6n11oavfs8d1214frjir9/f1/3efe2d7518cc476186e03d51278a4fd5 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_ 2023-04-03 02:54:59,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testGetAfterCompaction__tg40ijoujjnh927acleaqphcfn5h8h17pbsoo18ejst6n11oavfs8d1214frjir9/f1/3efe2d7518cc476186e03d51278a4fd5 into eeadc61284a31488b8fb04799304e6ab/f1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_ - updating store file list. 2023-04-03 02:54:59,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_ into eeadc61284a31488b8fb04799304e6ab/f1 2023-04-03 02:54:59,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testGetAfterCompaction__tg40ijoujjnh927acleaqphcfn5h8h17pbsoo18ejst6n11oavfs8d1214frjir9/f1/3efe2d7518cc476186e03d51278a4fd5 into eeadc61284a31488b8fb04799304e6ab/f1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_) 2023-04-03 02:54:59,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testGetAfterCompaction__tg40ijoujjnh927acleaqphcfn5h8h17pbsoo18ejst6n11oavfs8d1214frjir9/f1/3efe2d7518cc476186e03d51278a4fd5 2023-04-03 02:54:59,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): eeadc61284a31488b8fb04799304e6ab/f1 is initiating minor compaction 2023-04-03 02:54:59,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1ee1cd1a; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:59,363 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eeadc61284a31488b8fb04799304e6ab/f1 in testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:59,364 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp, totalSize=10.0 K 2023-04-03 02:54:59,364 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 759932c22db74b3eb912465e71581a9b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:54:59,364 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 7905d67abe1a48d0a040ea675fef851c_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:54:59,366 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testGetAfterCompaction' column='f1' 2023-04-03 02:54:59,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:59,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304038f11a113eb3045cf897b13b6d2cc252a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304038f11a113eb3045cf897b13b6d2cc252a 2023-04-03 02:54:59,371 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230403e899672510e04a8b946cae4ff870f57a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230403e899672510e04a8b946cae4ff870f57a 2023-04-03 02:54:59,371 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-04-03 02:54:59,371 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230403' 2023-04-03 02:54:59,372 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,372 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,372 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230403 for table='testGetAfterCompaction' and column='f1' 2023-04-03 02:54:59,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-04-03 02:54:59,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:54:59,374 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-03 02:54:59,377 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 513 msec 2023-04-03 02:54:59,388 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): eeadc61284a31488b8fb04799304e6ab#f1#compaction#76 average throughput is 0.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:54:59,399 DEBUG [Listener at localhost/37595] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-03 02:54:59,411 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50164, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-03 02:54:59,412 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45927] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:54:59,412 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-04-03 02:54:59,412 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-03 02:54:59,413 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45927] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-03 02:54:59,413 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45927] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. because compaction request was cancelled 2023-04-03 02:54:59,464 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,465 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,465 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,515 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,516 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,516 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,566 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,566 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,566 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,617 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,618 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,618 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,665 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-03 02:54:59,665 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-04-03 02:54:59,669 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,669 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,669 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,720 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,720 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,720 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,771 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,771 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,771 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,822 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498850; access_time=1680490498833; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,822 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1680490499292; access_time=1680490498883; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-03 02:54:59,823 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,830 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/.tmp/f1/c4cb8d927fa645b888d8cecb81aad189 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/c4cb8d927fa645b888d8cecb81aad189 2023-04-03 02:54:59,837 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in eeadc61284a31488b8fb04799304e6ab/f1 of eeadc61284a31488b8fb04799304e6ab into c4cb8d927fa645b888d8cecb81aad189(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:54:59,837 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:54:59,837 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1680490499363; duration=0sec 2023-04-03 02:54:59,837 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:54:59,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_] to archive 2023-04-03 02:54:59,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:54:59,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/759932c22db74b3eb912465e71581a9b 2023-04-03 02:54:59,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/7905d67abe1a48d0a040ea675fef851c_SeqId_12_ 2023-04-03 02:54:59,873 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1680490498817; access_time=1680490497997; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,873 INFO [Listener at localhost/37595] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/c4cb8d927fa645b888d8cecb81aad189; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1680490499822; access_time=1680490499370; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-03 02:54:59,996 INFO [Listener at localhost/37595] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-04-03 02:55:00,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-04-03 02:55:00,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,013 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490500013"}]},"ts":"1680490500013"} 2023-04-03 02:55:00,015 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-04-03 02:55:00,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-03 02:55:00,019 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-04-03 02:55:00,020 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, UNASSIGN}] 2023-04-03 02:55:00,022 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, UNASSIGN 2023-04-03 02:55:00,023 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=eeadc61284a31488b8fb04799304e6ab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,023 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680490500023"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490500023"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490500023"}]},"ts":"1680490500023"} 2023-04-03 02:55:00,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure eeadc61284a31488b8fb04799304e6ab, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:00,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-03 02:55:00,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:55:00,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eeadc61284a31488b8fb04799304e6ab, disabling compactions & flushes 2023-04-03 02:55:00,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:55:00,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:55:00,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. after waiting 1 ms 2023-04-03 02:55:00,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:55:00,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-04-03 02:55:00,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-03 02:55:00,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab. 2023-04-03 02:55:00,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eeadc61284a31488b8fb04799304e6ab: 2023-04-03 02:55:00,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:55:00,184 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=eeadc61284a31488b8fb04799304e6ab, regionState=CLOSED 2023-04-03 02:55:00,184 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680490500184"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490500184"}]},"ts":"1680490500184"} 2023-04-03 02:55:00,187 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-04-03 02:55:00,187 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure eeadc61284a31488b8fb04799304e6ab, server=jenkins-hbase4.apache.org,45927,1680490420719 in 161 msec 2023-04-03 02:55:00,188 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-04-03 02:55:00,188 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=eeadc61284a31488b8fb04799304e6ab, UNASSIGN in 167 msec 2023-04-03 02:55:00,189 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490500189"}]},"ts":"1680490500189"} 2023-04-03 02:55:00,190 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-04-03 02:55:00,193 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-04-03 02:55:00,194 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 189 msec 2023-04-03 02:55:00,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-03 02:55:00,318 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-04-03 02:55:00,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-04-03 02:55:00,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,334 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,335 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-04-03 02:55:00,352 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:55:00,355 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1, FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/recovered.edits] 2023-04-03 02:55:00,360 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/a39d316d50c449f983f45a14d7a7244f 2023-04-03 02:55:00,361 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/c4cb8d927fa645b888d8cecb81aad189 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/f1/c4cb8d927fa645b888d8cecb81aad189 2023-04-03 02:55:00,365 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/recovered.edits/15.seqid to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab/recovered.edits/15.seqid 2023-04-03 02:55:00,365 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testGetAfterCompaction/eeadc61284a31488b8fb04799304e6ab 2023-04-03 02:55:00,365 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-04-03 02:55:00,366 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-04-03 02:55:00,367 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-04-03 02:55:00,371 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023040351a2588bec53491d9316ef7842a48e75 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023040351a2588bec53491d9316ef7842a48e75 2023-04-03 02:55:00,372 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-04-03 02:55:00,375 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,405 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-04-03 02:55:00,410 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-04-03 02:55:00,411 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,411 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-04-03 02:55:00,412 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680490500412"}]},"ts":"9223372036854775807"} 2023-04-03 02:55:00,413 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-03 02:55:00,413 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => eeadc61284a31488b8fb04799304e6ab, NAME => 'testGetAfterCompaction,,1680490496870.eeadc61284a31488b8fb04799304e6ab.', STARTKEY => '', ENDKEY => ''}] 2023-04-03 02:55:00,414 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-04-03 02:55:00,414 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680490500414"}]},"ts":"9223372036854775807"} 2023-04-03 02:55:00,416 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-04-03 02:55:00,418 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-03 02:55:00,419 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 93 msec 2023-04-03 02:55:00,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-04-03 02:55:00,438 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-04-03 02:55:00,453 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=249 (was 241) - Thread LEAK? -, OpenFileDescriptor=629 (was 621) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 413), ProcessCount=170 (was 170), AvailableMemoryMB=3505 (was 3514) 2023-04-03 02:55:00,469 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=249, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=413, ProcessCount=170, AvailableMemoryMB=3504 2023-04-03 02:55:00,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-03 02:55:00,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-04-03 02:55:00,474 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:55:00,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-04-03 02:55:00,475 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:55:00,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-03 02:55:00,477 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,478 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,478 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,478 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e empty. 2023-04-03 02:55:00,478 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e empty. 2023-04-03 02:55:00,478 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a empty. 2023-04-03 02:55:00,479 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,479 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,479 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,479 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-04-03 02:55:00,498 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-04-03 02:55:00,500 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 65d190e4c51d6a4a46cb459041dddd1e, NAME => 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:00,500 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => a431e83bfacaea3fe93968c2ff1d8e5a, NAME => 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:00,501 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 1db75b42190724ca7b89184d5fdf0c8e, NAME => 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 65d190e4c51d6a4a46cb459041dddd1e, disabling compactions & flushes 2023-04-03 02:55:00,542 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. after waiting 0 ms 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,542 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,542 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing a431e83bfacaea3fe93968c2ff1d8e5a, disabling compactions & flushes 2023-04-03 02:55:00,546 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. after waiting 0 ms 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,546 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,546 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 1db75b42190724ca7b89184d5fdf0c8e, disabling compactions & flushes 2023-04-03 02:55:00,550 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. after waiting 0 ms 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,550 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,550 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:00,553 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:55:00,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500554"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490500554"}]},"ts":"1680490500554"} 2023-04-03 02:55:00,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490500554"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490500554"}]},"ts":"1680490500554"} 2023-04-03 02:55:00,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500554"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490500554"}]},"ts":"1680490500554"} 2023-04-03 02:55:00,557 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-03 02:55:00,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:55:00,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490500558"}]},"ts":"1680490500558"} 2023-04-03 02:55:00,559 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-04-03 02:55:00,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN}] 2023-04-03 02:55:00,566 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN 2023-04-03 02:55:00,566 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN 2023-04-03 02:55:00,566 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN 2023-04-03 02:55:00,567 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:00,567 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:00,567 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:00,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-03 02:55:00,719 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,719 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,719 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490500719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490500719"}]},"ts":"1680490500719"} 2023-04-03 02:55:00,719 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,719 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490500719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490500719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490500719"}]},"ts":"1680490500719"} 2023-04-03 02:55:00,719 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500719"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490500719"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490500719"}]},"ts":"1680490500719"} 2023-04-03 02:55:00,721 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:00,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:00,725 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:00,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-03 02:55:00,856 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:00,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a431e83bfacaea3fe93968c2ff1d8e5a, NAME => 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.', STARTKEY => '1', ENDKEY => '2'} 2023-04-03 02:55:00,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,878 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,893 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:00,893 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:00,894 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a431e83bfacaea3fe93968c2ff1d8e5a columnFamilyName family1 2023-04-03 02:55:00,897 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(310): Store=a431e83bfacaea3fe93968c2ff1d8e5a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,898 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,900 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2 2023-04-03 02:55:00,900 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2 2023-04-03 02:55:00,900 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a431e83bfacaea3fe93968c2ff1d8e5a columnFamilyName family2 2023-04-03 02:55:00,902 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(310): Store=a431e83bfacaea3fe93968c2ff1d8e5a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:00,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:00,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:00,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a431e83bfacaea3fe93968c2ff1d8e5a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66533869, jitterRate=-0.008568093180656433}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:00,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:00,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., pid=84, masterSystemTime=1680490500873 2023-04-03 02:55:00,917 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,917 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490500917"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490500917"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490500917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490500917"}]},"ts":"1680490500917"} 2023-04-03 02:55:00,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:00,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 65d190e4c51d6a4a46cb459041dddd1e, NAME => 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.', STARTKEY => '', ENDKEY => '1'} 2023-04-03 02:55:00,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,921 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-04-03 02:55:00,921 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719 in 197 msec 2023-04-03 02:55:00,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,922 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN in 359 msec 2023-04-03 02:55:00,925 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,927 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:00,927 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:00,927 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65d190e4c51d6a4a46cb459041dddd1e columnFamilyName family1 2023-04-03 02:55:00,928 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(310): Store=65d190e4c51d6a4a46cb459041dddd1e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,928 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,930 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2 2023-04-03 02:55:00,930 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2 2023-04-03 02:55:00,930 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65d190e4c51d6a4a46cb459041dddd1e columnFamilyName family2 2023-04-03 02:55:00,931 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(310): Store=65d190e4c51d6a4a46cb459041dddd1e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:00,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:00,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:00,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 65d190e4c51d6a4a46cb459041dddd1e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71339152, jitterRate=0.06303620338439941}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:00,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:00,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., pid=85, masterSystemTime=1680490500873 2023-04-03 02:55:00,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:00,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500944"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490500944"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490500944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490500944"}]},"ts":"1680490500944"} 2023-04-03 02:55:00,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1db75b42190724ca7b89184d5fdf0c8e, NAME => 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.', STARTKEY => '2', ENDKEY => ''} 2023-04-03 02:55:00,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:00,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,949 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN in 386 msec 2023-04-03 02:55:00,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-04-03 02:55:00,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 221 msec 2023-04-03 02:55:00,953 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,958 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:00,958 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:00,959 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1db75b42190724ca7b89184d5fdf0c8e columnFamilyName family1 2023-04-03 02:55:00,960 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(310): Store=1db75b42190724ca7b89184d5fdf0c8e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,960 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,963 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2 2023-04-03 02:55:00,963 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2 2023-04-03 02:55:00,964 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1db75b42190724ca7b89184d5fdf0c8e columnFamilyName family2 2023-04-03 02:55:00,965 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(310): Store=1db75b42190724ca7b89184d5fdf0c8e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:00,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:00,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:00,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:00,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1db75b42190724ca7b89184d5fdf0c8e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65340008, jitterRate=-0.02635800838470459}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:00,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:00,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., pid=83, masterSystemTime=1680490500873 2023-04-03 02:55:00,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:00,977 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:00,977 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490500977"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490500977"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490500977"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490500977"}]},"ts":"1680490500977"} 2023-04-03 02:55:00,980 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-04-03 02:55:00,980 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 257 msec 2023-04-03 02:55:00,981 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=79 2023-04-03 02:55:00,981 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN in 416 msec 2023-04-03 02:55:00,982 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:55:00,982 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490500982"}]},"ts":"1680490500982"} 2023-04-03 02:55:00,983 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-04-03 02:55:00,986 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:55:00,987 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 514 msec 2023-04-03 02:55:01,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-03 02:55:01,079 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-04-03 02:55:01,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:01,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:01,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:01,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:01,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-03 02:55:01,095 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-03 02:55:01,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:01,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-03 02:55:01,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-03 02:55:01,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,095 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:01,097 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:01,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:01,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:01,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:01,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:01,098 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:01,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:01,098 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-03 02:55:01,098 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-03 02:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-03 02:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:01,098 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-03 02:55:01,098 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:01,099 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:01,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a431e83bfacaea3fe93968c2ff1d8e5a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:01,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:01,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:01,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:01,105 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:01,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1db75b42190724ca7b89184d5fdf0c8e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:01,105 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:01,106 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 65d190e4c51d6a4a46cb459041dddd1e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:01,135 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 2023-04-03 02:55:01,143 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 2023-04-03 02:55:01,146 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/8b67d6f0eaba494286ae37d842b80352 2023-04-03 02:55:01,168 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/f1e39049cadc4a108baccdf6294d044b 2023-04-03 02:55:01,541 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 2023-04-03 02:55:01,590 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0 2023-04-03 02:55:01,596 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/85917e242a1b4e6f8444acd5c29d594f 2023-04-03 02:55:01,598 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee 2023-04-03 02:55:01,603 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/8b67d6f0eaba494286ae37d842b80352 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352 2023-04-03 02:55:01,606 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/7e893c05310a4d6d911a5f1089d47271 2023-04-03 02:55:01,612 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:01,613 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/85917e242a1b4e6f8444acd5c29d594f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f 2023-04-03 02:55:01,613 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/f1e39049cadc4a108baccdf6294d044b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b 2023-04-03 02:55:01,618 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:01,619 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a431e83bfacaea3fe93968c2ff1d8e5a in 515ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:01,619 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-04-03 02:55:01,619 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:01,620 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:01,620 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:01,620 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:01,621 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/7e893c05310a4d6d911a5f1089d47271 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271 2023-04-03 02:55:01,625 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:01,626 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1db75b42190724ca7b89184d5fdf0c8e in 521ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:01,627 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:01,627 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:01,627 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:01,948 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/6d98df1c60184cfaaa124a69bb2ea2e9 2023-04-03 02:55:01,964 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e 2023-04-03 02:55:01,968 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/5773d36c870a442180b04de860e76703 2023-04-03 02:55:01,972 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/6d98df1c60184cfaaa124a69bb2ea2e9 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9 2023-04-03 02:55:01,976 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:01,977 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/5773d36c870a442180b04de860e76703 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703 2023-04-03 02:55:01,981 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:01,981 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 65d190e4c51d6a4a46cb459041dddd1e in 875ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:01,982 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:01,982 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-03 02:55:01,982 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-03 02:55:01,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,983 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:01,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:01,984 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,984 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:01,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:01,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:01,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-03 02:55:01,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30d7d355[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:01,985 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-03 02:55:01,985 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,986 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:01,987 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,987 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:01,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-03 02:55:01,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-03 02:55:01,987 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:01,988 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:01,988 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-03 02:55:01,988 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:01,988 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-03 02:55:01,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:01,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:01,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:01,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:01,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-03 02:55:01,990 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-03 02:55:01,991 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:01,991 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:01,991 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-03 02:55:01,991 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:01,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:01,995 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,995 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:01,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:01,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:01,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:01,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:01,997 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:01,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:01,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:01,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:01,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:01,999 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:01,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:02,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-03 02:55:02,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:01,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:01,999 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:02,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-03 02:55:02,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:02,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:02,000 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:02,100 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:02,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:02,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:02,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:02,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:02,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:02,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:02,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-03 02:55:02,108 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-03 02:55:02,108 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:02,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-03 02:55:02,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-03 02:55:02,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:02,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:02,110 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:02,110 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:02,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:02,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:02,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:02,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:02,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:02,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-03 02:55:02,112 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-03 02:55:02,112 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-03 02:55:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-03 02:55:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-03 02:55:02,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:02,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:02,128 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a431e83bfacaea3fe93968c2ff1d8e5a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:02,129 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:02,129 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:02,129 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1db75b42190724ca7b89184d5fdf0c8e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:02,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:02,132 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:02,132 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:02,132 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:02,132 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 65d190e4c51d6a4a46cb459041dddd1e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:02,196 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 2023-04-03 02:55:02,591 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d 2023-04-03 02:55:02,608 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/f4774bef7f7b45e49dd6fa69652d4563 2023-04-03 02:55:02,612 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee 2023-04-03 02:55:03,014 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 2023-04-03 02:55:03,024 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/ed68c06dbd99442db5aff42ce68e23cd 2023-04-03 02:55:03,030 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b 2023-04-03 02:55:03,447 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9 2023-04-03 02:55:03,453 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/71a93b5ed2d24e6fa3981dc0d4504b2d 2023-04-03 02:55:03,457 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 2023-04-03 02:55:03,461 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:03,462 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/71a93b5ed2d24e6fa3981dc0d4504b2d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d 2023-04-03 02:55:03,464 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/9acebd5f73144c5cb35cd6bdad143f6f 2023-04-03 02:55:03,466 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:03,467 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 65d190e4c51d6a4a46cb459041dddd1e in 1335ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:03,467 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:03,467 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:03,467 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:03,468 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/f4774bef7f7b45e49dd6fa69652d4563 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563 2023-04-03 02:55:03,472 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:03,473 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/9acebd5f73144c5cb35cd6bdad143f6f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f 2023-04-03 02:55:03,478 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:03,479 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a431e83bfacaea3fe93968c2ff1d8e5a in 1351ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:03,479 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:03,479 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:03,479 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:03,482 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b 2023-04-03 02:55:03,888 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/47e8e37d6b8d4aed96bcc79fe101013a 2023-04-03 02:55:03,893 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/ed68c06dbd99442db5aff42ce68e23cd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd 2023-04-03 02:55:03,896 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:03,897 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/47e8e37d6b8d4aed96bcc79fe101013a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a 2023-04-03 02:55:03,901 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:03,902 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1db75b42190724ca7b89184d5fdf0c8e in 1773ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:03,902 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:03,902 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:03,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:03,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:03,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-03 02:55:03,902 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-03 02:55:03,904 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:03,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:03,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,904 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:03,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-03 02:55:03,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@676d181b[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:03,906 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-03 02:55:03,906 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,907 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-03 02:55:03,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-03 02:55:03,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:03,908 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,908 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:03,909 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-03 02:55:03,909 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:03,909 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-03 02:55:03,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:03,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:03,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:03,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:03,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:03,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:03,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-03 02:55:03,912 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-03 02:55:03,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:03,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:03,912 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-03 02:55:03,912 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:03,915 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,915 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:03,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:03,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,915 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:03,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:03,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:03,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:03,916 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:03,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,917 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:03,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:03,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:03,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:03,919 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:03,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-03 02:55:03,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:03,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:03,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:03,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:03,920 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:03,920 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:03,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-03 02:55:03,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:03,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:04,020 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:04,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:04,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:04,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:04,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:04,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:04,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:04,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-03 02:55:04,034 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-03 02:55:04,034 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:04,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-03 02:55:04,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-03 02:55:04,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:04,035 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:04,036 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:04,036 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:04,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:04,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:04,037 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:04,037 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:04,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:04,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:04,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-03 02:55:04,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-03 02:55:04,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-03 02:55:04,040 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-03 02:55:04,040 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:04,040 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-03 02:55:04,042 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:04,042 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:04,042 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a431e83bfacaea3fe93968c2ff1d8e5a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:04,043 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:04,043 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:04,043 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1db75b42190724ca7b89184d5fdf0c8e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:04,044 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:04,044 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:04,044 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:04,044 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:04,045 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 65d190e4c51d6a4a46cb459041dddd1e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:04,106 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa 2023-04-03 02:55:04,114 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/e1a2e9a5406e4581a151b79bc114d9d5 2023-04-03 02:55:04,136 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29 2023-04-03 02:55:04,142 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/8f09c6e18b424ff181c391767fc1bd6a 2023-04-03 02:55:04,156 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/e1a2e9a5406e4581a151b79bc114d9d5 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5 2023-04-03 02:55:04,163 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:04,165 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/8f09c6e18b424ff181c391767fc1bd6a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a 2023-04-03 02:55:04,170 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:04,171 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 65d190e4c51d6a4a46cb459041dddd1e in 126ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:04,171 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:04,172 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:04,172 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:04,493 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 2023-04-03 02:55:04,501 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e 2023-04-03 02:55:04,899 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 2023-04-03 02:55:04,912 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/886ca7a0c9184f7689ba28ae4e7b5e3f 2023-04-03 02:55:04,985 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28 2023-04-03 02:55:05,002 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/2ed38c058b6943c89bebded073e16732 2023-04-03 02:55:05,010 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/886ca7a0c9184f7689ba28ae4e7b5e3f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f 2023-04-03 02:55:05,015 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:05,016 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/2ed38c058b6943c89bebded073e16732 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732 2023-04-03 02:55:05,026 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:05,027 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a431e83bfacaea3fe93968c2ff1d8e5a in 985ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:05,027 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:05,027 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:05,027 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:05,366 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe 2023-04-03 02:55:05,385 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/fb825f81bafd47c0870f9fb4db1c1aaa 2023-04-03 02:55:05,391 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 2023-04-03 02:55:05,395 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:05,396 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/fb825f81bafd47c0870f9fb4db1c1aaa as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa 2023-04-03 02:55:05,400 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:05,401 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1db75b42190724ca7b89184d5fdf0c8e in 1358ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:05,401 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:05,401 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-03 02:55:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-03 02:55:05,403 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:05,403 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:05,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-03 02:55:05,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@469c94d1[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:05,405 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-03 02:55:05,405 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,407 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-03 02:55:05,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-03 02:55:05,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:05,408 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,408 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:05,411 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-03 02:55:05,411 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,411 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:05,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,411 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-03 02:55:05,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:05,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-03 02:55:05,414 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-03 02:55:05,414 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:05,414 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:05,414 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-03 02:55:05,414 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:05,416 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,416 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,416 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:05,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,417 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,418 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:05,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:05,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,423 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:05,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-03 02:55:05,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:05,423 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:05,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:05,424 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:05,424 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:05,424 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:05,424 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,524 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:05,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:05,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:05,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:05,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:05,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:05,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:05,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-03 02:55:05,539 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-03 02:55:05,539 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:05,539 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-03 02:55:05,539 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-03 02:55:05,539 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,539 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:05,541 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:05,541 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:05,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:05,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:05,541 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:05,542 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:05,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:05,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:05,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:05,542 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-03 02:55:05,543 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-03 02:55:05,543 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-03 02:55:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-03 02:55:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:05,543 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-03 02:55:05,544 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:05,544 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:05,545 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a431e83bfacaea3fe93968c2ff1d8e5a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:05,544 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:05,555 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:05,555 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1db75b42190724ca7b89184d5fdf0c8e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:05,560 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:05,560 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:05,560 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:05,560 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:05,560 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 65d190e4c51d6a4a46cb459041dddd1e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:05,652 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab 2023-04-03 02:55:06,048 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc 2023-04-03 02:55:06,053 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f 2023-04-03 02:55:06,055 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/d88ebf74ee774be4b1f8c4fe37ee07fe 2023-04-03 02:55:06,058 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca 2023-04-03 02:55:06,086 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d 2023-04-03 02:55:06,086 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4 2023-04-03 02:55:06,122 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/d99c992a13a94d3497fd4958f8b237ef 2023-04-03 02:55:06,127 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca 2023-04-03 02:55:06,131 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:06,132 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/d99c992a13a94d3497fd4958f8b237ef as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef 2023-04-03 02:55:06,138 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:06,138 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a431e83bfacaea3fe93968c2ff1d8e5a in 593ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:06,138 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:06,139 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:06,139 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:06,378 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:06,459 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/964dff66dbdc4c0aab1d3a792990cf2d 2023-04-03 02:55:06,513 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/d647b4cce0e84a5d94ae6e0188f6bc19 2023-04-03 02:55:06,518 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/d88ebf74ee774be4b1f8c4fe37ee07fe as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe 2023-04-03 02:55:06,523 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:06,524 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/d647b4cce0e84a5d94ae6e0188f6bc19 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19 2023-04-03 02:55:06,531 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:06,532 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1db75b42190724ca7b89184d5fdf0c8e in 977ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:06,532 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:06,532 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:06,885 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78 2023-04-03 02:55:06,890 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/980e6943e1e94c7189b57b8d7efbf065 2023-04-03 02:55:06,895 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/964dff66dbdc4c0aab1d3a792990cf2d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d 2023-04-03 02:55:06,898 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:06,899 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/980e6943e1e94c7189b57b8d7efbf065 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065 2023-04-03 02:55:06,904 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:06,904 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 65d190e4c51d6a4a46cb459041dddd1e in 1344ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:06,904 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:06,904 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:06,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:06,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:06,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-03 02:55:06,905 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-03 02:55:06,906 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,906 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:06,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:06,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,907 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:06,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:06,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:06,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:06,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-03 02:55:06,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36d63b04[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:06,908 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-03 02:55:06,908 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,909 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-03 02:55:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-03 02:55:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:06,910 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,910 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:06,911 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-03 02:55:06,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:06,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:06,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:06,911 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-03 02:55:06,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:06,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:06,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:06,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:06,913 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-03 02:55:06,913 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:06,913 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:06,913 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-03 02:55:06,913 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:06,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-03 02:55:06,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:06,916 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:06,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:06,917 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:06,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:06,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:06,918 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:06,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:06,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:06,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:06,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-03 02:55:06,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:06,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:06,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:06,919 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:06,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:06,919 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:06,920 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:06,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:07,020 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:07,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:07,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:07,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1680490507032 (current time:1680490507032). 2023-04-03 02:55:07,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-04-03 02:55:07,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] zookeeper.ReadOnlyZKClient(139): Connect 0x33bd46f4 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:07,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@501f7533, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:07,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:07,047 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55362, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:07,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33bd46f4 to 127.0.0.1:52880 2023-04-03 02:55:07,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:07,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-04-03 02:55:07,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:07,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:07,060 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:07,061 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1680491107061 2023-04-03 02:55:07,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:07,071 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1680490507021 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-04-03 02:55:07,073 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:07,074 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-03 02:55:07,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 14 msec 2023-04-03 02:55:07,077 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-03 02:55:07,077 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:07,077 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-04-03 02:55:07,077 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1680491107077 2023-04-03 02:55:07,084 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1680490507021 2023-04-03 02:55:07,084 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1680490507021' 2023-04-03 02:55:07,084 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-04-03 02:55:07,084 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1680490507021' starting 'acquire' 2023-04-03 02:55:07,084 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1680490507021', kicking off acquire phase on members. 2023-04-03 02:55:07,085 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,085 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1680490507021 2023-04-03 02:55:07,087 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-04-03 02:55:07,087 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-04-03 02:55:07,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-03 02:55:07,087 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,087 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:07,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1680490507021 2023-04-03 02:55:07,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-04-03 02:55:07,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1680490507021 2023-04-03 02:55:07,088 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1680490507021 from table testCompactionWithHFileLink type FLUSH 2023-04-03 02:55:07,090 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1680490507021 2023-04-03 02:55:07,090 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1680490507021' with timeout 300000ms 2023-04-03 02:55:07,090 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-04-03 02:55:07,091 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1680490507021' starting 'acquire' stage 2023-04-03 02:55:07,091 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1680490507021' locally acquired 2023-04-03 02:55:07,091 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (snaptb-1680490507021) in zk 2023-04-03 02:55:07,095 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-03 02:55:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1680490507021' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:07,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:07,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:07,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'snaptb-1680490507021' on coordinator 2023-04-03 02:55:07,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38151bf0[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:07,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1680490507021' starting 'in-barrier' execution. 2023-04-03 02:55:07,097 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,098 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,098 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1680490507021' received 'reached' from coordinator. 2023-04-03 02:55:07,099 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,099 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:07,099 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:07,100 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:07,101 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.' region-info for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,101 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.' region-info for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,101 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.' region-info for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,107 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-03 02:55:07,107 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-03 02:55:07,107 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-03 02:55:07,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca] hfiles 2023-04-03 02:55:07,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe] hfiles 2023-04-03 02:55:07,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d] hfiles 2023-04-03 02:55:07,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,112 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,115 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef] hfiles 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19] hfiles 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065] hfiles 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732 for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,116 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. completed. 2023-04-03 02:55:07,128 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-04-03 02:55:07,140 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. completed. 2023-04-03 02:55:07,141 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:07,141 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-04-03 02:55:07,177 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:07,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:07,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:07,182 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-04-03 02:55:07,382 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:07,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:07,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:07,383 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-04-03 02:55:07,534 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. completed. 2023-04-03 02:55:07,534 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-04-03 02:55:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-04-03 02:55:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1680490507021' locally completed 2023-04-03 02:55:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1680490507021' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1680490507021' has notified controller of completion 2023-04-03 02:55:07,536 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-03 02:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1680490507021' completed. 2023-04-03 02:55:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:07,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:07,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1680490507021' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:07,540 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'snaptb-1680490507021', counting down latch. Waiting for 0 more 2023-04-03 02:55:07,540 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1680490507021' execution completed 2023-04-03 02:55:07,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:07,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:07,540 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1680490507021 2023-04-03 02:55:07,541 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1680490507021including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-04-03 02:55:07,543 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,543 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:07,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-03 02:55:07,543 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-04-03 02:55:07,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-04-03 02:55:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:07,544 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-03 02:55:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,545 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1680490507021 2023-04-03 02:55:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,547 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,548 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-04-03 02:55:07,548 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1680490507021 2023-04-03 02:55:07,548 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1680490507021 2023-04-03 02:55:07,548 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:07,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-04-03 02:55:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-03 02:55:07,548 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1680490507021 2023-04-03 02:55:07,548 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-04-03 02:55:07,548 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1680490507021 2023-04-03 02:55:07,548 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-04-03 02:55:07,548 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1680490507021/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:07,548 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-04-03 02:55:07,549 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-04-03 02:55:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-03 02:55:07,549 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1680490507021 2023-04-03 02:55:07,549 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1680490507021 2023-04-03 02:55:07,549 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d] mob files 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f 2023-04-03 02:55:07,550 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9] mob files 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e 2023-04-03 02:55:07,551 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9 2023-04-03 02:55:07,570 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1680490507021 2023-04-03 02:55:07,571 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/snaptb-1680490507021 2023-04-03 02:55:07,659 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/snaptb-1680490507021 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/snaptb-1680490507021 2023-04-03 02:55:07,684 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:07,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:07,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:07,685 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-04-03 02:55:08,127 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1680490507021 of table testCompactionWithHFileLink completed 2023-04-03 02:55:08,127 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/snaptb-1680490507021 2023-04-03 02:55:08,128 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/snaptb-1680490507021 2023-04-03 02:55:08,128 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1680490507021 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1680490507072Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1680490507569 (+497 ms)Consolidate snapshot: snaptb-1680490507021 at 1680490507569Loading Region manifests for snaptb-1680490507021 at 1680490507570 (+1 ms)Writing data manifest for snaptb-1680490507021 at 1680490507591 (+21 ms)Verifying snapshot: snaptb-1680490507021 at 1680490507610 (+19 ms)Snapshot snaptb-1680490507021 of table testCompactionWithHFileLink completed at 1680490508127 (+517 ms) 2023-04-03 02:55:08,129 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-03 02:55:08,130 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.0560 sec 2023-04-03 02:55:08,185 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:08,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:08,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-04-03 02:55:08,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:08,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:08,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-03 02:55:08,230 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-03 02:55:08,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:08,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-03 02:55:08,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-03 02:55:08,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:08,230 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:08,231 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:08,231 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:08,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:08,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:08,231 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:08,231 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:08,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:08,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:08,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:08,232 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-03 02:55:08,232 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-03 02:55:08,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-03 02:55:08,233 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-03 02:55:08,233 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:08,233 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-03 02:55:08,233 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:08,233 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. started... 2023-04-03 02:55:08,233 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:08,233 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. started... 2023-04-03 02:55:08,233 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a431e83bfacaea3fe93968c2ff1d8e5a 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:08,233 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1db75b42190724ca7b89184d5fdf0c8e 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:08,236 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:08,236 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:08,236 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:08,236 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. started... 2023-04-03 02:55:08,236 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 65d190e4c51d6a4a46cb459041dddd1e 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:08,299 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/81fd2296ce6a481d8b7def3aac7df2cb 2023-04-03 02:55:08,303 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81fd2296ce6a481d8b7def3aac7df2cb 2023-04-03 02:55:08,680 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/198cbaabb5524236b52e18997292d654 2023-04-03 02:55:08,684 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 198cbaabb5524236b52e18997292d654 2023-04-03 02:55:08,696 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/cc53b64deac64f4e8e085acd905c141b 2023-04-03 02:55:08,702 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc53b64deac64f4e8e085acd905c141b 2023-04-03 02:55:08,723 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/d3c5399537e94221af73f72c0a84847f 2023-04-03 02:55:08,725 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/54a25ecf41d34297bd824d8812c5451e 2023-04-03 02:55:08,728 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3c5399537e94221af73f72c0a84847f 2023-04-03 02:55:08,729 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/cc53b64deac64f4e8e085acd905c141b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b 2023-04-03 02:55:08,730 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54a25ecf41d34297bd824d8812c5451e 2023-04-03 02:55:08,733 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/81fd2296ce6a481d8b7def3aac7df2cb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb 2023-04-03 02:55:08,736 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc53b64deac64f4e8e085acd905c141b 2023-04-03 02:55:08,737 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b, entries=3, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:08,738 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/d3c5399537e94221af73f72c0a84847f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f 2023-04-03 02:55:08,742 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81fd2296ce6a481d8b7def3aac7df2cb 2023-04-03 02:55:08,742 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb, entries=3, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:08,742 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/54a25ecf41d34297bd824d8812c5451e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e 2023-04-03 02:55:08,749 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3c5399537e94221af73f72c0a84847f 2023-04-03 02:55:08,749 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f, entries=1, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:08,750 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54a25ecf41d34297bd824d8812c5451e 2023-04-03 02:55:08,750 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e, entries=1, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:08,750 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 65d190e4c51d6a4a46cb459041dddd1e in 514ms, sequenceid=24, compaction requested=true 2023-04-03 02:55:08,750 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a431e83bfacaea3fe93968c2ff1d8e5a in 517ms, sequenceid=24, compaction requested=true 2023-04-03 02:55:08,750 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:08,750 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:08,750 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:08,750 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:08,750 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:09,096 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/bdef8a2c543b4cd19f66f2b2218a7452 2023-04-03 02:55:09,100 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bdef8a2c543b4cd19f66f2b2218a7452 2023-04-03 02:55:09,101 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/198cbaabb5524236b52e18997292d654 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654 2023-04-03 02:55:09,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 198cbaabb5524236b52e18997292d654 2023-04-03 02:55:09,105 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654, entries=3, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:09,106 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/bdef8a2c543b4cd19f66f2b2218a7452 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452 2023-04-03 02:55:09,110 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bdef8a2c543b4cd19f66f2b2218a7452 2023-04-03 02:55:09,110 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452, entries=1, sequenceid=24, filesize=5.0 K 2023-04-03 02:55:09,111 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1db75b42190724ca7b89184d5fdf0c8e in 878ms, sequenceid=24, compaction requested=true 2023-04-03 02:55:09,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:09,111 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-03 02:55:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-03 02:55:09,113 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,113 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:09,113 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,114 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:09,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-03 02:55:09,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37b77129[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:09,115 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-03 02:55:09,115 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,117 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-03 02:55:09,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-03 02:55:09,118 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:09,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:09,122 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,122 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-03 02:55:09,122 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:09,122 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-03 02:55:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:09,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:09,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:09,124 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-03 02:55:09,124 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-03 02:55:09,124 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:09,124 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:09,124 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-03 02:55:09,124 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:09,126 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,126 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:09,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,126 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:09,126 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:09,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:09,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,127 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:09,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-03 02:55:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:09,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,131 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-03 02:55:09,132 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-03 02:55:09,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:09,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-03 02:55:09,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:09,132 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:09,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:09,132 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:09,132 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:09,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-03 02:55:09,232 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:09,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:09,234 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:09,234 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): a431e83bfacaea3fe93968c2ff1d8e5a/family1 is initiating major compaction (all files) 2023-04-03 02:55:09,234 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:09,234 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:09,235 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of a431e83bfacaea3fe93968c2ff1d8e5a/family1 in testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:09,235 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp, totalSize=26.7 K 2023-04-03 02:55:09,236 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 8b67d6f0eaba494286ae37d842b80352, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501088 2023-04-03 02:55:09,237 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting f4774bef7f7b45e49dd6fa69652d4563, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:09,237 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 886ca7a0c9184f7689ba28ae4e7b5e3f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:09,237 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 0e65b4f45bf649b1bf9cbefd4a4c8eca, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:09,238 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 81fd2296ce6a481d8b7def3aac7df2cb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:09,256 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): a431e83bfacaea3fe93968c2ff1d8e5a#family1#compaction#107 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:09,306 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del 2023-04-03 02:55:09,307 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del 2023-04-03 02:55:09,665 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-03 02:55:09,763 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec1bae5f44bf47ca96585b2665ac794a 2023-04-03 02:55:09,764 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/ec1bae5f44bf47ca96585b2665ac794a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/ec1bae5f44bf47ca96585b2665ac794a 2023-04-03 02:55:09,767 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ec1bae5f44bf47ca96585b2665ac794a 2023-04-03 02:55:09,768 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a431e83bfacaea3fe93968c2ff1d8e5a/family1 of a431e83bfacaea3fe93968c2ff1d8e5a into ec1bae5f44bf47ca96585b2665ac794a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:09,768 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:09,768 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:09,768 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): a431e83bfacaea3fe93968c2ff1d8e5a/family2 is initiating major compaction (all files) 2023-04-03 02:55:09,768 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:09,768 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:09,769 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of a431e83bfacaea3fe93968c2ff1d8e5a/family2 in testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:09,769 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp, totalSize=25.4 K 2023-04-03 02:55:09,770 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 85917e242a1b4e6f8444acd5c29d594f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501088 2023-04-03 02:55:09,770 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 9acebd5f73144c5cb35cd6bdad143f6f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:09,770 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 2ed38c058b6943c89bebded073e16732, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:09,771 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d99c992a13a94d3497fd4958f8b237ef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:09,771 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 54a25ecf41d34297bd824d8812c5451e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:09,781 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): a431e83bfacaea3fe93968c2ff1d8e5a#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:09,838 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb] to archive 2023-04-03 02:55:09,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:09,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/8b67d6f0eaba494286ae37d842b80352 2023-04-03 02:55:09,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f4774bef7f7b45e49dd6fa69652d4563 2023-04-03 02:55:09,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/886ca7a0c9184f7689ba28ae4e7b5e3f 2023-04-03 02:55:09,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/0e65b4f45bf649b1bf9cbefd4a4c8eca 2023-04-03 02:55:09,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/81fd2296ce6a481d8b7def3aac7df2cb 2023-04-03 02:55:10,190 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del 2023-04-03 02:55:10,191 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del 2023-04-03 02:55:11,003 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 41928ddb23db48dc8528238f355cae04 2023-04-03 02:55:11,004 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/41928ddb23db48dc8528238f355cae04 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/41928ddb23db48dc8528238f355cae04 2023-04-03 02:55:11,007 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 41928ddb23db48dc8528238f355cae04 2023-04-03 02:55:11,008 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a431e83bfacaea3fe93968c2ff1d8e5a/family2 of a431e83bfacaea3fe93968c2ff1d8e5a into 41928ddb23db48dc8528238f355cae04(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:11,008 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:11,008 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:11,008 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 1db75b42190724ca7b89184d5fdf0c8e/family1 is initiating major compaction (all files) 2023-04-03 02:55:11,008 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:11,008 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:11,009 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 1db75b42190724ca7b89184d5fdf0c8e/family1 in testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:11,009 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp, totalSize=26.7 K 2023-04-03 02:55:11,009 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting f1e39049cadc4a108baccdf6294d044b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501089 2023-04-03 02:55:11,009 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting ed68c06dbd99442db5aff42ce68e23cd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:11,010 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 5af47a6c3b5b4b5498a4e50b6c97b8d0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:11,010 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d88ebf74ee774be4b1f8c4fe37ee07fe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:11,010 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 198cbaabb5524236b52e18997292d654, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:11,019 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 1db75b42190724ca7b89184d5fdf0c8e#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:11,037 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e] to archive 2023-04-03 02:55:11,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:11,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/85917e242a1b4e6f8444acd5c29d594f 2023-04-03 02:55:11,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/9acebd5f73144c5cb35cd6bdad143f6f 2023-04-03 02:55:11,041 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/2ed38c058b6943c89bebded073e16732 2023-04-03 02:55:11,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/d99c992a13a94d3497fd4958f8b237ef 2023-04-03 02:55:11,043 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/54a25ecf41d34297bd824d8812c5451e 2023-04-03 02:55:11,426 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del 2023-04-03 02:55:11,426 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del 2023-04-03 02:55:12,239 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ae4e47b9340d43c2a5fa4647942e1c68 2023-04-03 02:55:12,240 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/ae4e47b9340d43c2a5fa4647942e1c68 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ae4e47b9340d43c2a5fa4647942e1c68 2023-04-03 02:55:12,243 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ae4e47b9340d43c2a5fa4647942e1c68 2023-04-03 02:55:12,244 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1db75b42190724ca7b89184d5fdf0c8e/family1 of 1db75b42190724ca7b89184d5fdf0c8e into ae4e47b9340d43c2a5fa4647942e1c68(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:12,244 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:12,244 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:12,244 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 1db75b42190724ca7b89184d5fdf0c8e/family2 is initiating major compaction (all files) 2023-04-03 02:55:12,244 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:12,244 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:12,245 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 1db75b42190724ca7b89184d5fdf0c8e/family2 in testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:12,245 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp, totalSize=25.4 K 2023-04-03 02:55:12,245 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 7e893c05310a4d6d911a5f1089d47271, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501089 2023-04-03 02:55:12,246 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 47e8e37d6b8d4aed96bcc79fe101013a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:12,246 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting fb825f81bafd47c0870f9fb4db1c1aaa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:12,246 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d647b4cce0e84a5d94ae6e0188f6bc19, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:12,246 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting bdef8a2c543b4cd19f66f2b2218a7452, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:12,256 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 1db75b42190724ca7b89184d5fdf0c8e#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:12,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654] to archive 2023-04-03 02:55:12,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:12,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/f1e39049cadc4a108baccdf6294d044b 2023-04-03 02:55:12,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ed68c06dbd99442db5aff42ce68e23cd 2023-04-03 02:55:12,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/5af47a6c3b5b4b5498a4e50b6c97b8d0 2023-04-03 02:55:12,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/d88ebf74ee774be4b1f8c4fe37ee07fe 2023-04-03 02:55:12,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/198cbaabb5524236b52e18997292d654 2023-04-03 02:55:12,663 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del 2023-04-03 02:55:12,664 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del 2023-04-03 02:55:13,477 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccf8133d47be4e0caf702605d10bcff8 2023-04-03 02:55:13,478 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/ccf8133d47be4e0caf702605d10bcff8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/ccf8133d47be4e0caf702605d10bcff8 2023-04-03 02:55:13,482 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccf8133d47be4e0caf702605d10bcff8 2023-04-03 02:55:13,482 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1db75b42190724ca7b89184d5fdf0c8e/family2 of 1db75b42190724ca7b89184d5fdf0c8e into ccf8133d47be4e0caf702605d10bcff8(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:13,483 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:13,483 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:13,483 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 65d190e4c51d6a4a46cb459041dddd1e/family1 is initiating major compaction (all files) 2023-04-03 02:55:13,483 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:13,483 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:13,483 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 65d190e4c51d6a4a46cb459041dddd1e/family1 in testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:13,483 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp, totalSize=26.7 K 2023-04-03 02:55:13,483 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 6d98df1c60184cfaaa124a69bb2ea2e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501086 2023-04-03 02:55:13,484 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting f4fcdb18893f4b7bab3fdfcb27f5afe6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:13,484 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting e1a2e9a5406e4581a151b79bc114d9d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:13,484 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 964dff66dbdc4c0aab1d3a792990cf2d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:13,485 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting cc53b64deac64f4e8e085acd905c141b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:13,494 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 65d190e4c51d6a4a46cb459041dddd1e#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:13,502 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del 2023-04-03 02:55:13,502 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del 2023-04-03 02:55:13,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452] to archive 2023-04-03 02:55:13,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:13,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/7e893c05310a4d6d911a5f1089d47271 2023-04-03 02:55:13,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/47e8e37d6b8d4aed96bcc79fe101013a 2023-04-03 02:55:13,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/fb825f81bafd47c0870f9fb4db1c1aaa 2023-04-03 02:55:13,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/d647b4cce0e84a5d94ae6e0188f6bc19 2023-04-03 02:55:13,544 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/bdef8a2c543b4cd19f66f2b2218a7452 2023-04-03 02:55:13,913 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e94ba9336d134742b684874c7020698b 2023-04-03 02:55:13,914 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/e94ba9336d134742b684874c7020698b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e94ba9336d134742b684874c7020698b 2023-04-03 02:55:13,917 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e94ba9336d134742b684874c7020698b 2023-04-03 02:55:13,918 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 65d190e4c51d6a4a46cb459041dddd1e/family1 of 65d190e4c51d6a4a46cb459041dddd1e into e94ba9336d134742b684874c7020698b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:13,918 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:13,918 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:13,918 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 65d190e4c51d6a4a46cb459041dddd1e/family2 is initiating major compaction (all files) 2023-04-03 02:55:13,919 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:13,919 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:13,919 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 65d190e4c51d6a4a46cb459041dddd1e/family2 in testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:13,919 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp, totalSize=25.4 K 2023-04-03 02:55:13,919 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 5773d36c870a442180b04de860e76703, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501086 2023-04-03 02:55:13,920 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 71a93b5ed2d24e6fa3981dc0d4504b2d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:13,920 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 8f09c6e18b424ff181c391767fc1bd6a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:13,920 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 980e6943e1e94c7189b57b8d7efbf065, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:13,920 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d3c5399537e94221af73f72c0a84847f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-03 02:55:13,931 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 65d190e4c51d6a4a46cb459041dddd1e#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:13,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b] to archive 2023-04-03 02:55:13,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:13,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/6d98df1c60184cfaaa124a69bb2ea2e9 2023-04-03 02:55:13,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f4fcdb18893f4b7bab3fdfcb27f5afe6 2023-04-03 02:55:13,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e1a2e9a5406e4581a151b79bc114d9d5 2023-04-03 02:55:13,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/964dff66dbdc4c0aab1d3a792990cf2d 2023-04-03 02:55:13,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/cc53b64deac64f4e8e085acd905c141b 2023-04-03 02:55:14,340 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del 2023-04-03 02:55:14,341 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del 2023-04-03 02:55:14,757 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac2330c833fe49b09b655c00a485afb0 2023-04-03 02:55:14,758 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/ac2330c833fe49b09b655c00a485afb0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/ac2330c833fe49b09b655c00a485afb0 2023-04-03 02:55:14,761 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac2330c833fe49b09b655c00a485afb0 2023-04-03 02:55:14,762 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 65d190e4c51d6a4a46cb459041dddd1e/family2 of 65d190e4c51d6a4a46cb459041dddd1e into ac2330c833fe49b09b655c00a485afb0(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:14,762 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:14,767 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:14,770 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:14,775 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:14,776 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:14,781 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:55:14,810 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-04-03 02:55:14,813 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del 2023-04-03 02:55:14,816 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del 2023-04-03 02:55:14,819 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del 2023-04-03 02:55:14,819 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:14,819 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(139): Connect 0x3ed89200 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:14,824 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e0a523f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:14,824 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:14,825 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:14,825 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:14,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f] to archive 2023-04-03 02:55:14,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:14,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/5773d36c870a442180b04de860e76703 2023-04-03 02:55:14,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/71a93b5ed2d24e6fa3981dc0d4504b2d 2023-04-03 02:55:14,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/8f09c6e18b424ff181c391767fc1bd6a 2023-04-03 02:55:14,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/980e6943e1e94c7189b57b8d7efbf065 2023-04-03 02:55:14,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/d3c5399537e94221af73f72c0a84847f 2023-04-03 02:55:15,168 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:15,340 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-04-03 02:55:15,340 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:55:15,340 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x112c6f7a to 127.0.0.1:52880 2023-04-03 02:55:15,340 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:15,341 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22469053 to 127.0.0.1:52880 2023-04-03 02:55:15,341 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:15,341 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:55:15,341 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x06eedba3 to 127.0.0.1:52880 2023-04-03 02:55:15,341 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:15,345 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21 2023-04-03 02:55:15,347 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,350 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017 2023-04-03 02:55:15,351 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,445 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:15,448 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56048, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:15,455 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:55:15,457 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46320, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:55:15,471 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/c513011c82474e26bb5a3f04e37049dc first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-04-03 02:55:15,471 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/22716707af274bf9b2a80d0c8a4db24e first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-04-03 02:55:15,473 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/c513011c82474e26bb5a3f04e37049dc}] 2023-04-03 02:55:15,473 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/22716707af274bf9b2a80d0c8a4db24e}] 2023-04-03 02:55:15,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/c513011c82474e26bb5a3f04e37049dc for inclusion in a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:15,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/22716707af274bf9b2a80d0c8a4db24e for inclusion in 65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last=1 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:15,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/22716707af274bf9b2a80d0c8a4db24e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__4u26fcso7jf2bds519mkhlu72o86foaeich5o0q4kfncd31fmj7i147jfas9dkeu/family1/22716707af274bf9b2a80d0c8a4db24e 2023-04-03 02:55:15,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/c513011c82474e26bb5a3f04e37049dc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__to8fps7kpuuimk2o5g29dtahh2fo314be7k5v2fnf4m5hqk2l5vp6svfen5ajph7/family1/c513011c82474e26bb5a3f04e37049dc 2023-04-03 02:55:15,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__4u26fcso7jf2bds519mkhlu72o86foaeich5o0q4kfncd31fmj7i147jfas9dkeu/family1/22716707af274bf9b2a80d0c8a4db24e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ 2023-04-03 02:55:15,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__to8fps7kpuuimk2o5g29dtahh2fo314be7k5v2fnf4m5hqk2l5vp6svfen5ajph7/family1/c513011c82474e26bb5a3f04e37049dc as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ 2023-04-03 02:55:15,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__4u26fcso7jf2bds519mkhlu72o86foaeich5o0q4kfncd31fmj7i147jfas9dkeu/family1/22716707af274bf9b2a80d0c8a4db24e into 65d190e4c51d6a4a46cb459041dddd1e/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ - updating store file list. 2023-04-03 02:55:15,483 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__to8fps7kpuuimk2o5g29dtahh2fo314be7k5v2fnf4m5hqk2l5vp6svfen5ajph7/family1/c513011c82474e26bb5a3f04e37049dc into a431e83bfacaea3fe93968c2ff1d8e5a/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ - updating store file list. 2023-04-03 02:55:15,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ into 65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:15,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__4u26fcso7jf2bds519mkhlu72o86foaeich5o0q4kfncd31fmj7i147jfas9dkeu/family1/22716707af274bf9b2a80d0c8a4db24e into 65d190e4c51d6a4a46cb459041dddd1e/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_) 2023-04-03 02:55:15,486 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ into a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:15,486 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__to8fps7kpuuimk2o5g29dtahh2fo314be7k5v2fnf4m5hqk2l5vp6svfen5ajph7/family1/c513011c82474e26bb5a3f04e37049dc into a431e83bfacaea3fe93968c2ff1d8e5a/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_) 2023-04-03 02:55:15,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__4u26fcso7jf2bds519mkhlu72o86foaeich5o0q4kfncd31fmj7i147jfas9dkeu/family1/22716707af274bf9b2a80d0c8a4db24e 2023-04-03 02:55:15,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__to8fps7kpuuimk2o5g29dtahh2fo314be7k5v2fnf4m5hqk2l5vp6svfen5ajph7/family1/c513011c82474e26bb5a3f04e37049dc 2023-04-03 02:55:15,488 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,489 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,489 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,490 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,490 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa 2023-04-03 02:55:15,491 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e 2023-04-03 02:55:15,491 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 2023-04-03 02:55:15,492 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 2023-04-03 02:55:15,492 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f 2023-04-03 02:55:15,493 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 2023-04-03 02:55:15,493 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d 2023-04-03 02:55:15,493 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:15,493 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403' 2023-04-03 02:55:15,494 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:15,494 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab 2023-04-03 02:55:15,494 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:15,494 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403' 2023-04-03 02:55:15,494 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:15,750 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8 2023-04-03 02:55:15,751 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,773 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/87d283af071b4ecbb04740e6e91ace6d first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-04-03 02:55:15,774 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/87d283af071b4ecbb04740e6e91ace6d}] 2023-04-03 02:55:15,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/87d283af071b4ecbb04740e6e91ace6d for inclusion in 1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:15,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-04-03 02:55:15,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-03 02:55:15,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:15,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/87d283af071b4ecbb04740e6e91ace6d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__ndcqf53ceu4rfnm1tdh75ld30k479cucttu8m79978inbtmbnpa9bgn8g9io1kk3/family1/87d283af071b4ecbb04740e6e91ace6d 2023-04-03 02:55:15,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__ndcqf53ceu4rfnm1tdh75ld30k479cucttu8m79978inbtmbnpa9bgn8g9io1kk3/family1/87d283af071b4ecbb04740e6e91ace6d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ 2023-04-03 02:55:15,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__ndcqf53ceu4rfnm1tdh75ld30k479cucttu8m79978inbtmbnpa9bgn8g9io1kk3/family1/87d283af071b4ecbb04740e6e91ace6d into 1db75b42190724ca7b89184d5fdf0c8e/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ - updating store file list. 2023-04-03 02:55:15,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ into 1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:15,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__ndcqf53ceu4rfnm1tdh75ld30k479cucttu8m79978inbtmbnpa9bgn8g9io1kk3/family1/87d283af071b4ecbb04740e6e91ace6d into 1db75b42190724ca7b89184d5fdf0c8e/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_) 2023-04-03 02:55:15,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__ndcqf53ceu4rfnm1tdh75ld30k479cucttu8m79978inbtmbnpa9bgn8g9io1kk3/family1/87d283af071b4ecbb04740e6e91ace6d 2023-04-03 02:55:15,785 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,785 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,786 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc 2023-04-03 02:55:15,787 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 2023-04-03 02:55:15,788 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 2023-04-03 02:55:15,789 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee 2023-04-03 02:55:15,789 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:15,789 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403' 2023-04-03 02:55:15,789 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:15,789 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:15,790 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:15,790 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del] 2023-04-03 02:55:15,790 DEBUG [Listener at localhost/37595] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,791 DEBUG [Listener at localhost/37595] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304036e0d5da211734276a970448d8a444d96_del 2023-04-03 02:55:15,791 INFO [Listener at localhost/37595] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-03 02:55:15,791 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del] 2023-04-03 02:55:15,791 DEBUG [Listener at localhost/37595] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,792 DEBUG [Listener at localhost/37595] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403c8c817fcfc534011943ff2bda0278f72_del 2023-04-03 02:55:15,792 INFO [Listener at localhost/37595] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-03 02:55:15,792 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del] 2023-04-03 02:55:15,793 DEBUG [Listener at localhost/37595] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:15,794 DEBUG [Listener at localhost/37595] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304038c6d5426290245709dfe6d7389d4762a_del 2023-04-03 02:55:15,794 INFO [Listener at localhost/37595] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-03 02:55:15,796 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:15,798 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:15,801 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:15,801 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:15,806 INFO [Listener at localhost/37595] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-04-03 02:55:15,807 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-04-03 02:55:15,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-04-03 02:55:15,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-03 02:55:15,810 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490515810"}]},"ts":"1680490515810"} 2023-04-03 02:55:15,811 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-04-03 02:55:15,814 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-04-03 02:55:15,815 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, UNASSIGN}] 2023-04-03 02:55:15,816 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, UNASSIGN 2023-04-03 02:55:15,816 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, UNASSIGN 2023-04-03 02:55:15,816 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, UNASSIGN 2023-04-03 02:55:15,817 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:15,817 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:15,817 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:15,817 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490515817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490515817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490515817"}]},"ts":"1680490515817"} 2023-04-03 02:55:15,817 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490515817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490515817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490515817"}]},"ts":"1680490515817"} 2023-04-03 02:55:15,817 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490515817"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490515817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490515817"}]},"ts":"1680490515817"} 2023-04-03 02:55:15,818 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:15,818 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:15,819 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:15,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-03 02:55:15,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:15,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 65d190e4c51d6a4a46cb459041dddd1e, disabling compactions & flushes 2023-04-03 02:55:15,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:15,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:15,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. after waiting 0 ms 2023-04-03 02:55:15,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:15,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-03 02:55:15,975 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:15,975 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:15,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:15,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:15,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1db75b42190724ca7b89184d5fdf0c8e, disabling compactions & flushes 2023-04-03 02:55:15,976 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:15,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:15,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. after waiting 0 ms 2023-04-03 02:55:15,976 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:15,977 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=CLOSED 2023-04-03 02:55:15,977 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490515976"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490515976"}]},"ts":"1680490515976"} 2023-04-03 02:55:15,979 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-04-03 02:55:15,979 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 160 msec 2023-04-03 02:55:15,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-03 02:55:15,980 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, UNASSIGN in 164 msec 2023-04-03 02:55:15,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:15,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:15,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:15,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:15,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a431e83bfacaea3fe93968c2ff1d8e5a, disabling compactions & flushes 2023-04-03 02:55:15,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:15,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:15,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. after waiting 0 ms 2023-04-03 02:55:15,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:15,982 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=CLOSED 2023-04-03 02:55:15,983 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490515982"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490515982"}]},"ts":"1680490515982"} 2023-04-03 02:55:15,985 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-04-03 02:55:15,985 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 165 msec 2023-04-03 02:55:15,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-03 02:55:15,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, UNASSIGN in 170 msec 2023-04-03 02:55:15,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:15,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:15,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:15,989 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=CLOSED 2023-04-03 02:55:15,989 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490515989"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490515989"}]},"ts":"1680490515989"} 2023-04-03 02:55:15,991 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-04-03 02:55:15,991 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719 in 172 msec 2023-04-03 02:55:15,992 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-04-03 02:55:15,992 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, UNASSIGN in 176 msec 2023-04-03 02:55:15,993 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490515993"}]},"ts":"1680490515993"} 2023-04-03 02:55:15,994 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-04-03 02:55:15,995 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-04-03 02:55:15,997 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 189 msec 2023-04-03 02:55:16,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-03 02:55:16,112 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-04-03 02:55:16,122 INFO [Listener at localhost/37595] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:16,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:16,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1680490516122 (current time:1680490516122). 2023-04-03 02:55:16,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-04-03 02:55:16,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] zookeeper.ReadOnlyZKClient(139): Connect 0x4d67b1de to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:16,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@571e6a30, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:16,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:16,130 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:16,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d67b1de to 127.0.0.1:52880 2023-04-03 02:55:16,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:16,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-04-03 02:55:16,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:16,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:16,134 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:16,135 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1680491116135 2023-04-03 02:55:16,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:16,135 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1680490507021-1680490516122 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-04-03 02:55:16,135 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:16,135 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-04-03 02:55:16,235 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:16,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:16,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:16,237 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-04-03 02:55:16,437 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:16,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:16,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:16,438 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-04-03 02:55:16,548 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-04-03 02:55:16,549 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-03 02:55:16,549 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-03 02:55:16,550 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/ec1bae5f44bf47ca96585b2665ac794a] hfiles 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e94ba9336d134742b684874c7020698b] hfiles 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ae4e47b9340d43c2a5fa4647942e1c68] hfiles 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ 2023-04-03 02:55:16,552 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ae4e47b9340d43c2a5fa4647942e1c68 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e94ba9336d134742b684874c7020698b 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/ec1bae5f44bf47ca96585b2665ac794a 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017] hfiles 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8 2023-04-03 02:55:16,553 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017 2023-04-03 02:55:16,554 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/ac2330c833fe49b09b655c00a485afb0] hfiles 2023-04-03 02:55:16,554 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/ac2330c833fe49b09b655c00a485afb0 2023-04-03 02:55:16,554 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/41928ddb23db48dc8528238f355cae04] hfiles 2023-04-03 02:55:16,554 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/41928ddb23db48dc8528238f355cae04 2023-04-03 02:55:16,557 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/ccf8133d47be4e0caf702605d10bcff8] hfiles 2023-04-03 02:55:16,557 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/ccf8133d47be4e0caf702605d10bcff8 2023-04-03 02:55:16,560 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9] hfiles 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304032c8f3caa11d447eb9c204ed65ac8ce2b 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023040335631154e5a446f4a0be433a1a251be0 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403cbc0557446904a04b0595d886de9df28 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230403fb1d1552ec78471eb52dfecc290845e4 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403122e0af6953b499bb519f43eea5626ee 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032bc588d6b9254b2ea0d122b5de57bfbe 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304032d8885cc5ff94ad98cb84b5e185f3f7d 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040382f438c65dc548df8dfd375cd1411d1b 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023040350b55ffb13ee4535a4e376ec1c436a29 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a690adee638f402e9b502bd370fd6b78 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403a76b57051011449bb6735506e77f9e4e 2023-04-03 02:55:16,561 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230403c1d6040de699453ba8f54cde9ddac0a9 2023-04-03 02:55:16,738 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:16,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:16,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:16,739 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-04-03 02:55:16,969 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-04-03 02:55:16,969 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:16,970 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:17,240 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:17,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:17,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-03 02:55:17,241 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-04-03 02:55:17,392 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1680490507021-1680490516122 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:18,207 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1680490507021-1680490516122 of table testCompactionWithHFileLink completed 2023-04-03 02:55:18,207 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:18,207 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:18,208 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1680490507021-1680490516122 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1680490516135Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1680490516548 (+413 ms)Consolidate snapshot: hbase-failsafe-snaptb-1680490507021-1680490516122 at 1680490516969 (+421 ms)Loading Region manifests for hbase-failsafe-snaptb-1680490507021-1680490516122 at 1680490516969Writing data manifest for hbase-failsafe-snaptb-1680490507021-1680490516122 at 1680490516972 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1680490507021-1680490516122 at 1680490517377 (+405 ms)Snapshot hbase-failsafe-snaptb-1680490507021-1680490516122 of table testCompactionWithHFileLink completed at 1680490518207 (+830 ms) 2023-04-03 02:55:18,209 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-03 02:55:18,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0760 sec 2023-04-03 02:55:18,241 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-03 02:55:18,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-03 02:55:18,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1680490507021-1680490516122 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-04-03 02:55:18,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1680490507021" table: "testCompactionWithHFileLink" creation_time: 1680490507032 type: FLUSH version: 2 ttl: 0 ) 2023-04-03 02:55:18,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1680490507021 as table=testCompactionWithHFileLink 2023-04-03 02:55:18,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-03 02:55:18,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-03 02:55:18,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-03 02:55:18,694 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-04-03 02:55:18,695 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-04-03 02:55:18,696 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-04-03 02:55:18,696 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-03 02:55:18,701 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1680490507021" table: "testCompactionWithHFileLink" creation_time: 1680490507032 type: FLUSH version: 2 ttl: 0 2023-04-03 02:55:18,701 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink 2023-04-03 02:55:18,706 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-04-03 02:55:18,709 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403813893bc3a5441bc8fdb2cfd840eb017 2023-04-03 02:55:18,710 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403fb8298d5dde74f8c84fcf12a16745a21 2023-04-03 02:55:18,711 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230403b4cc8e7e86e246d585948ac3d5a65de8 2023-04-03 02:55:18,711 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,714 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,716 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,719 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,722 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,725 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,727 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,730 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,733 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,735 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,738 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,741 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d of snapshot=snaptb-1680490507021 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-03 02:55:18,746 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230403be7d2094de9744248b74ab1b7396bd1b_del 2023-04-03 02:55:18,747 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304038e0341c533864a5f8747abd421ca5d8d_del 2023-04-03 02:55:18,748 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304034cad97253a6a460192279631cc6e1a25_del 2023-04-03 02:55:18,748 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:18,748 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:18,748 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:18,751 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/49a425cbdf4a449791a81e0d3d5e73d2_SeqId_29_ 2023-04-03 02:55:18,751 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/7139130088d14e8b9437154a45257a1e_SeqId_29_ 2023-04-03 02:55:18,751 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/4cfe9de7278a4297a1f64555ab18d794_SeqId_29_ 2023-04-03 02:55:18,752 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ae4e47b9340d43c2a5fa4647942e1c68 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/ae4e47b9340d43c2a5fa4647942e1c68 2023-04-03 02:55:18,753 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f1e39049cadc4a108baccdf6294d044b of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,753 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e94ba9336d134742b684874c7020698b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/e94ba9336d134742b684874c7020698b 2023-04-03 02:55:18,753 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6d98df1c60184cfaaa124a69bb2ea2e9 of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,753 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/ec1bae5f44bf47ca96585b2665ac794a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/ec1bae5f44bf47ca96585b2665ac794a 2023-04-03 02:55:18,753 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8b67d6f0eaba494286ae37d842b80352 of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,760 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f4fcdb18893f4b7bab3fdfcb27f5afe6 of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,760 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f4774bef7f7b45e49dd6fa69652d4563 of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,761 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ed68c06dbd99442db5aff42ce68e23cd of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,763 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e1a2e9a5406e4581a151b79bc114d9d5 of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,763 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 886ca7a0c9184f7689ba28ae4e7b5e3f of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,765 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5af47a6c3b5b4b5498a4e50b6c97b8d0 of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,768 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 964dff66dbdc4c0aab1d3a792990cf2d of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,768 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0e65b4f45bf649b1bf9cbefd4a4c8eca of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,769 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d88ebf74ee774be4b1f8c4fe37ee07fe of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,775 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/ac2330c833fe49b09b655c00a485afb0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/ac2330c833fe49b09b655c00a485afb0 2023-04-03 02:55:18,775 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5773d36c870a442180b04de860e76703 of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,775 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/41928ddb23db48dc8528238f355cae04 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/41928ddb23db48dc8528238f355cae04 2023-04-03 02:55:18,775 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 85917e242a1b4e6f8444acd5c29d594f of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,776 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/ccf8133d47be4e0caf702605d10bcff8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/ccf8133d47be4e0caf702605d10bcff8 2023-04-03 02:55:18,776 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7e893c05310a4d6d911a5f1089d47271 of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,778 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9acebd5f73144c5cb35cd6bdad143f6f of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,779 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 47e8e37d6b8d4aed96bcc79fe101013a of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,779 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 71a93b5ed2d24e6fa3981dc0d4504b2d of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,782 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2ed38c058b6943c89bebded073e16732 of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,782 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8f09c6e18b424ff181c391767fc1bd6a of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,782 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fb825f81bafd47c0870f9fb4db1c1aaa of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,785 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d99c992a13a94d3497fd4958f8b237ef of snapshot=snaptb-1680490507021 to region=a431e83bfacaea3fe93968c2ff1d8e5a table=testCompactionWithHFileLink 2023-04-03 02:55:18,785 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d647b4cce0e84a5d94ae6e0188f6bc19 of snapshot=snaptb-1680490507021 to region=1db75b42190724ca7b89184d5fdf0c8e table=testCompactionWithHFileLink 2023-04-03 02:55:18,785 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 980e6943e1e94c7189b57b8d7efbf065 of snapshot=snaptb-1680490507021 to region=65d190e4c51d6a4a46cb459041dddd1e table=testCompactionWithHFileLink 2023-04-03 02:55:18,788 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1680490507021" table: "testCompactionWithHFileLink" creation_time: 1680490507032 type: FLUSH version: 2 ttl: 0 2023-04-03 02:55:18,789 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680490518789"}]},"ts":"9223372036854775807"} 2023-04-03 02:55:18,789 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680490518789"}]},"ts":"9223372036854775807"} 2023-04-03 02:55:18,789 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680490518789"}]},"ts":"9223372036854775807"} 2023-04-03 02:55:18,791 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-03 02:55:18,791 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1db75b42190724ca7b89184d5fdf0c8e, NAME => 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 65d190e4c51d6a4a46cb459041dddd1e, NAME => 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => a431e83bfacaea3fe93968c2ff1d8e5a, NAME => 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.', STARTKEY => '1', ENDKEY => '2'}] 2023-04-03 02:55:18,791 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490518790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490518790"}]},"ts":"1680490518790"} 2023-04-03 02:55:18,791 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490518790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490518790"}]},"ts":"1680490518790"} 2023-04-03 02:55:18,791 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490518790"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490518790"}]},"ts":"1680490518790"} 2023-04-03 02:55:18,793 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-03 02:55:18,793 INFO [PEWorker-5] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-04-03 02:55:18,793 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 1db75b42190724ca7b89184d5fdf0c8e, NAME => 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 65d190e4c51d6a4a46cb459041dddd1e, NAME => 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => a431e83bfacaea3fe93968c2ff1d8e5a, NAME => 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.', STARTKEY => '1', ENDKEY => '2'}] 2023-04-03 02:55:18,793 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1680490507021 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-04-03 02:55:18,795 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1680490507021" table: "testCompactionWithHFileLink" creation_time: 1680490507032 type: FLUSH version: 2 ttl: 0 ) in 518 msec 2023-04-03 02:55:18,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-03 02:55:18,890 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-04-03 02:55:18,890 INFO [Listener at localhost/37595] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:18,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1680490507021-1680490516122" 2023-04-03 02:55:18,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1680490507021-1680490516122 2023-04-03 02:55:18,899 INFO [Listener at localhost/37595] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-04-03 02:55:18,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-04-03 02:55:18,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-04-03 02:55:18,911 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-04-03 02:55:18,911 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490518911"}]},"ts":"1680490518911"} 2023-04-03 02:55:18,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-03 02:55:18,912 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-04-03 02:55:18,915 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-04-03 02:55:18,915 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN}] 2023-04-03 02:55:18,916 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN 2023-04-03 02:55:18,916 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN 2023-04-03 02:55:18,916 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN 2023-04-03 02:55:18,917 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-03 02:55:18,917 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-03 02:55:18,917 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-03 02:55:19,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-03 02:55:19,068 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,068 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,068 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,068 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490519068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490519068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490519068"}]},"ts":"1680490519068"} 2023-04-03 02:55:19,068 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490519068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490519068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490519068"}]},"ts":"1680490519068"} 2023-04-03 02:55:19,068 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490519068"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490519068"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490519068"}]},"ts":"1680490519068"} 2023-04-03 02:55:19,070 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:19,070 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:19,071 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:19,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-03 02:55:19,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:19,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 65d190e4c51d6a4a46cb459041dddd1e, NAME => 'testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.', STARTKEY => '', ENDKEY => '1'} 2023-04-03 02:55:19,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:19,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,225 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,226 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:19,226 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:19,227 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65d190e4c51d6a4a46cb459041dddd1e columnFamilyName family1 2023-04-03 02:55:19,237 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9 2023-04-03 02:55:19,242 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d 2023-04-03 02:55:19,247 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5 2023-04-03 02:55:19,252 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6 2023-04-03 02:55:19,252 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(310): Store=65d190e4c51d6a4a46cb459041dddd1e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,252 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,253 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2 2023-04-03 02:55:19,253 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2 2023-04-03 02:55:19,253 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 65d190e4c51d6a4a46cb459041dddd1e columnFamilyName family2 2023-04-03 02:55:19,259 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703 2023-04-03 02:55:19,263 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d 2023-04-03 02:55:19,267 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a 2023-04-03 02:55:19,271 DEBUG [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065 2023-04-03 02:55:19,272 INFO [StoreOpener-65d190e4c51d6a4a46cb459041dddd1e-1] regionserver.HStore(310): Store=65d190e4c51d6a4a46cb459041dddd1e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:19,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:55:19,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 65d190e4c51d6a4a46cb459041dddd1e; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65719451, jitterRate=-0.020703867077827454}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:19,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:19,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., pid=102, masterSystemTime=1680490519221 2023-04-03 02:55:19,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:19,278 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-03 02:55:19,278 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-03 02:55:19,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:19,279 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:19,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:19,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:19,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1db75b42190724ca7b89184d5fdf0c8e, NAME => 'testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.', STARTKEY => '2', ENDKEY => ''} 2023-04-03 02:55:19,280 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=65d190e4c51d6a4a46cb459041dddd1e, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:19,280 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490519280"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490519280"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490519280"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490519280"}]},"ts":"1680490519280"} 2023-04-03 02:55:19,280 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-03 02:55:19,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,280 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1912): 65d190e4c51d6a4a46cb459041dddd1e/family1 is initiating minor compaction (all files) 2023-04-03 02:55:19,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,280 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-03 02:55:19,280 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 65d190e4c51d6a4a46cb459041dddd1e/family1 in testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:19,280 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1912): 65d190e4c51d6a4a46cb459041dddd1e/family2 is initiating minor compaction (all files) 2023-04-03 02:55:19,280 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp, totalSize=21.6 K 2023-04-03 02:55:19,280 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 65d190e4c51d6a4a46cb459041dddd1e/family2 in testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:55:19,280 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp, totalSize=20.4 K 2023-04-03 02:55:19,281 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501086 2023-04-03 02:55:19,281 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501086 2023-04-03 02:55:19,282 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:19,282 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:19,283 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:19,283 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:19,283 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-04-03 02:55:19,283 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 65d190e4c51d6a4a46cb459041dddd1e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 211 msec 2023-04-03 02:55:19,283 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:19,284 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:19,284 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=65d190e4c51d6a4a46cb459041dddd1e, ASSIGN in 368 msec 2023-04-03 02:55:19,292 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,293 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:19,294 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:19,294 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1db75b42190724ca7b89184d5fdf0c8e columnFamilyName family1 2023-04-03 02:55:19,312 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0 2023-04-03 02:55:19,322 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe 2023-04-03 02:55:19,328 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] throttle.PressureAwareThroughputController(145): 65d190e4c51d6a4a46cb459041dddd1e#family2#compaction#113 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:19,329 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 65d190e4c51d6a4a46cb459041dddd1e#family1#compaction#114 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:19,331 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd 2023-04-03 02:55:19,338 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b 2023-04-03 02:55:19,338 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(310): Store=1db75b42190724ca7b89184d5fdf0c8e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,338 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,339 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2 2023-04-03 02:55:19,339 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2 2023-04-03 02:55:19,339 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1db75b42190724ca7b89184d5fdf0c8e columnFamilyName family2 2023-04-03 02:55:19,345 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a 2023-04-03 02:55:19,349 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271 2023-04-03 02:55:19,357 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19 2023-04-03 02:55:19,362 DEBUG [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa 2023-04-03 02:55:19,362 INFO [StoreOpener-1db75b42190724ca7b89184d5fdf0c8e-1] regionserver.HStore(310): Store=1db75b42190724ca7b89184d5fdf0c8e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:19,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:55:19,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1db75b42190724ca7b89184d5fdf0c8e; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67253109, jitterRate=0.002149417996406555}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:19,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:19,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., pid=101, masterSystemTime=1680490519221 2023-04-03 02:55:19,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-03 02:55:19,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-03 02:55:19,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:19,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:19,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:19,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a431e83bfacaea3fe93968c2ff1d8e5a, NAME => 'testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.', STARTKEY => '1', ENDKEY => '2'} 2023-04-03 02:55:19,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,370 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=1db75b42190724ca7b89184d5fdf0c8e, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:19,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,370 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1680490519370"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490519370"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490519370"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490519370"}]},"ts":"1680490519370"} 2023-04-03 02:55:19,371 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,372 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:19,372 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:19,372 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a431e83bfacaea3fe93968c2ff1d8e5a columnFamilyName family1 2023-04-03 02:55:19,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-04-03 02:55:19,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 1db75b42190724ca7b89184d5fdf0c8e, server=jenkins-hbase4.apache.org,45927,1680490420719 in 301 msec 2023-04-03 02:55:19,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1db75b42190724ca7b89184d5fdf0c8e, ASSIGN in 458 msec 2023-04-03 02:55:19,382 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca 2023-04-03 02:55:19,387 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f 2023-04-03 02:55:19,393 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352 2023-04-03 02:55:19,398 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563 2023-04-03 02:55:19,398 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(310): Store=a431e83bfacaea3fe93968c2ff1d8e5a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,399 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,399 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2 2023-04-03 02:55:19,399 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2 2023-04-03 02:55:19,400 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a431e83bfacaea3fe93968c2ff1d8e5a columnFamilyName family2 2023-04-03 02:55:19,406 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732 2023-04-03 02:55:19,412 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f 2023-04-03 02:55:19,417 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f 2023-04-03 02:55:19,422 DEBUG [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(539): loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef 2023-04-03 02:55:19,422 INFO [StoreOpener-a431e83bfacaea3fe93968c2ff1d8e5a-1] regionserver.HStore(310): Store=a431e83bfacaea3fe93968c2ff1d8e5a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:19,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:19,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:55:19,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a431e83bfacaea3fe93968c2ff1d8e5a; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59167385, jitterRate=-0.11833725869655609}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:19,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:19,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., pid=103, masterSystemTime=1680490519221 2023-04-03 02:55:19,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-04-03 02:55:19,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-04-03 02:55:19,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:19,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:19,429 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=a431e83bfacaea3fe93968c2ff1d8e5a, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:19,429 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490519428"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490519428"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490519428"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490519428"}]},"ts":"1680490519428"} 2023-04-03 02:55:19,431 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-04-03 02:55:19,431 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure a431e83bfacaea3fe93968c2ff1d8e5a, server=jenkins-hbase4.apache.org,45927,1680490420719 in 359 msec 2023-04-03 02:55:19,432 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-04-03 02:55:19,433 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a431e83bfacaea3fe93968c2ff1d8e5a, ASSIGN in 516 msec 2023-04-03 02:55:19,433 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490519433"}]},"ts":"1680490519433"} 2023-04-03 02:55:19,434 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-04-03 02:55:19,435 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-04-03 02:55:19,437 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 531 msec 2023-04-03 02:55:19,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-03 02:55:19,515 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-04-03 02:55:19,527 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:19,533 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:19,541 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:19,542 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:19,547 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:55:19,559 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-04-03 02:55:19,559 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:19,559 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(139): Connect 0x28b43b23 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:19,563 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28189b88, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:19,563 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:19,563 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:19,563 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:20,025 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304037f6085580576418d866d8f8cfaeacbab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304037f6085580576418d866d8f8cfaeacbab 2023-04-03 02:55:20,026 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,040 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:20,044 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56068, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:20,051 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:55:20,054 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46324, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:55:20,068 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/9b851a0578114b0c9eda5ee16976b2d0 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-04-03 02:55:20,068 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/9b851a0578114b0c9eda5ee16976b2d0}] 2023-04-03 02:55:20,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/9b851a0578114b0c9eda5ee16976b2d0 for inclusion in 1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:20,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-04-03 02:55:20,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-03 02:55:20,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:20,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403/family1/9b851a0578114b0c9eda5ee16976b2d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__p19e09i8jif3gv8mmi1602bqhcdi7m8540kfjeri38os6egqaafm9mgptqs6j8tm/family1/9b851a0578114b0c9eda5ee16976b2d0 2023-04-03 02:55:20,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__p19e09i8jif3gv8mmi1602bqhcdi7m8540kfjeri38os6egqaafm9mgptqs6j8tm/family1/9b851a0578114b0c9eda5ee16976b2d0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_ 2023-04-03 02:55:20,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__p19e09i8jif3gv8mmi1602bqhcdi7m8540kfjeri38os6egqaafm9mgptqs6j8tm/family1/9b851a0578114b0c9eda5ee16976b2d0 into 1db75b42190724ca7b89184d5fdf0c8e/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_ - updating store file list. 2023-04-03 02:55:20,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_ into 1db75b42190724ca7b89184d5fdf0c8e/family1 2023-04-03 02:55:20,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__p19e09i8jif3gv8mmi1602bqhcdi7m8540kfjeri38os6egqaafm9mgptqs6j8tm/family1/9b851a0578114b0c9eda5ee16976b2d0 into 1db75b42190724ca7b89184d5fdf0c8e/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_) 2023-04-03 02:55:20,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__p19e09i8jif3gv8mmi1602bqhcdi7m8540kfjeri38os6egqaafm9mgptqs6j8tm/family1/9b851a0578114b0c9eda5ee16976b2d0 2023-04-03 02:55:20,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:20,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-04-03 02:55:20,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 1db75b42190724ca7b89184d5fdf0c8e/family1 is initiating minor compaction (all files) 2023-04-03 02:55:20,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@79df6429; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-03 02:55:20,087 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,087 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:20,089 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202304030cc7958fc9434425949300053d4dd9dc 2023-04-03 02:55:20,090 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040336438e50876b4b1c9218be00811f11d9 2023-04-03 02:55:20,091 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202304033a6c01c4ba8643a5b32f987da7d307e3 2023-04-03 02:55:20,092 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040345973add71184450b1594cb3e4f92cee 2023-04-03 02:55:20,092 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:20,092 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230403' 2023-04-03 02:55:20,092 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:20,147 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family1/8914f67cde31470996479882c845b969 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/8914f67cde31470996479882c845b969 2023-04-03 02:55:20,151 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-04-03 02:55:20,152 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 65d190e4c51d6a4a46cb459041dddd1e/family1 of 65d190e4c51d6a4a46cb459041dddd1e into 8914f67cde31470996479882c845b969(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:20,152 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:20,152 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., storeName=65d190e4c51d6a4a46cb459041dddd1e/family1, priority=12, startTime=1680490519277; duration=0sec 2023-04-03 02:55:20,152 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-03 02:55:20,152 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1db75b42190724ca7b89184d5fdf0c8e/family1 in testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:20,152 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp, totalSize=28.9 K 2023-04-03 02:55:20,153 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501089 2023-04-03 02:55:20,153 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:20,153 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:20,154 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:20,154 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 030005f1742743109cd2d18d3555172c_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1680490501089 2023-04-03 02:55:20,166 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1db75b42190724ca7b89184d5fdf0c8e#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:20,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d] to archive 2023-04-03 02:55:20,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:20,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-6d98df1c60184cfaaa124a69bb2ea2e9 2023-04-03 02:55:20,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-f4fcdb18893f4b7bab3fdfcb27f5afe6 2023-04-03 02:55:20,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-e1a2e9a5406e4581a151b79bc114d9d5 2023-04-03 02:55:20,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-964dff66dbdc4c0aab1d3a792990cf2d 2023-04-03 02:55:20,423 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403c7de78380b13436a8d746bc2c4082c4f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230403c7de78380b13436a8d746bc2c4082c4f 2023-04-03 02:55:20,423 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403770881bd4a7f43cf9d1a092d26558307 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230403770881bd4a7f43cf9d1a092d26558307 2023-04-03 02:55:20,424 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,425 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,459 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/a2467838de304c95828fd08f4750cdc5 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-04-03 02:55:20,459 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/981756c57ddf4cff85f89106b4999501 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-04-03 02:55:20,460 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/a2467838de304c95828fd08f4750cdc5}] 2023-04-03 02:55:20,460 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=32 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/981756c57ddf4cff85f89106b4999501}] 2023-04-03 02:55:20,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/a2467838de304c95828fd08f4750cdc5 for inclusion in a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:20,463 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/981756c57ddf4cff85f89106b4999501 for inclusion in 65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:20,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-04-03 02:55:20,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-04-03 02:55:20,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last=1 2023-04-03 02:55:20,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-03 02:55:20,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:20,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:20,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403/family1/981756c57ddf4cff85f89106b4999501 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__m7f0n352p2uneecit13d45dn5vhps9r26uks5h9oa3l9a3lqsnvscqc8669v00oi/family1/981756c57ddf4cff85f89106b4999501 2023-04-03 02:55:20,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403/family1/a2467838de304c95828fd08f4750cdc5 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__v7o446qfhqcjc7ma7avljipu1dci86rhre01ijobkfk30jruoonmmpo7ba3qgh9s/family1/a2467838de304c95828fd08f4750cdc5 2023-04-03 02:55:20,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__m7f0n352p2uneecit13d45dn5vhps9r26uks5h9oa3l9a3lqsnvscqc8669v00oi/family1/981756c57ddf4cff85f89106b4999501 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f7aa68aa9ecd4fbcbeb7de5af4965841_SeqId_35_ 2023-04-03 02:55:20,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__v7o446qfhqcjc7ma7avljipu1dci86rhre01ijobkfk30jruoonmmpo7ba3qgh9s/family1/a2467838de304c95828fd08f4750cdc5 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_ 2023-04-03 02:55:20,473 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__m7f0n352p2uneecit13d45dn5vhps9r26uks5h9oa3l9a3lqsnvscqc8669v00oi/family1/981756c57ddf4cff85f89106b4999501 into 65d190e4c51d6a4a46cb459041dddd1e/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f7aa68aa9ecd4fbcbeb7de5af4965841_SeqId_35_ - updating store file list. 2023-04-03 02:55:20,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__v7o446qfhqcjc7ma7avljipu1dci86rhre01ijobkfk30jruoonmmpo7ba3qgh9s/family1/a2467838de304c95828fd08f4750cdc5 into a431e83bfacaea3fe93968c2ff1d8e5a/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_ - updating store file list. 2023-04-03 02:55:20,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f7aa68aa9ecd4fbcbeb7de5af4965841_SeqId_35_ into 65d190e4c51d6a4a46cb459041dddd1e/family1 2023-04-03 02:55:20,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_ into a431e83bfacaea3fe93968c2ff1d8e5a/family1 2023-04-03 02:55:20,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__m7f0n352p2uneecit13d45dn5vhps9r26uks5h9oa3l9a3lqsnvscqc8669v00oi/family1/981756c57ddf4cff85f89106b4999501 into 65d190e4c51d6a4a46cb459041dddd1e/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family1/f7aa68aa9ecd4fbcbeb7de5af4965841_SeqId_35_) 2023-04-03 02:55:20,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__v7o446qfhqcjc7ma7avljipu1dci86rhre01ijobkfk30jruoonmmpo7ba3qgh9s/family1/a2467838de304c95828fd08f4750cdc5 into a431e83bfacaea3fe93968c2ff1d8e5a/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_) 2023-04-03 02:55:20,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__m7f0n352p2uneecit13d45dn5vhps9r26uks5h9oa3l9a3lqsnvscqc8669v00oi/family1/981756c57ddf4cff85f89106b4999501 2023-04-03 02:55:20,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testCompactionWithHFileLink__v7o446qfhqcjc7ma7avljipu1dci86rhre01ijobkfk30jruoonmmpo7ba3qgh9s/family1/a2467838de304c95828fd08f4750cdc5 2023-04-03 02:55:20,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:20,480 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,481 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:20,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-04-03 02:55:20,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): a431e83bfacaea3fe93968c2ff1d8e5a/family1 is initiating minor compaction (all files) 2023-04-03 02:55:20,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@8db99e6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-03 02:55:20,483 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023040393d89d7f10f445b49b3cc6f9576035fa 2023-04-03 02:55:20,484 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,484 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403b38de38b14a34f17a67acc7e2c4a4ab0 2023-04-03 02:55:20,485 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:20,485 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403dfa201e5cdc64fd586f68d78b4fd6e5f 2023-04-03 02:55:20,486 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304037051e094332c4cd69ff409d81379225e 2023-04-03 02:55:20,486 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230403ed9b97c6edb248608481e0bc70f5e32d 2023-04-03 02:55:20,486 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:20,487 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230403' 2023-04-03 02:55:20,487 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:20,487 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403a47d7aa6e668482fac3e51424687be07 2023-04-03 02:55:20,489 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403da8599e607814ac0a496133b754e7b10 2023-04-03 02:55:20,490 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230403dd052e3d2c3d4336bcbc822d60d535ab 2023-04-03 02:55:20,490 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-03 02:55:20,490 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230403' 2023-04-03 02:55:20,490 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230403 for table='testCompactionWithHFileLink' and column='family1' 2023-04-03 02:55:20,490 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-04-03 02:55:20,495 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:20,497 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:20,500 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:20,501 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:20,507 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:20,508 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:20,527 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=288 (was 249) - Thread LEAK? -, OpenFileDescriptor=759 (was 629) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=407 (was 413), ProcessCount=170 (was 170), AvailableMemoryMB=3397 (was 3504) 2023-04-03 02:55:20,540 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=288, OpenFileDescriptor=759, MaxFileDescriptor=60000, SystemLoadAverage=407, ProcessCount=170, AvailableMemoryMB=3396 2023-04-03 02:55:20,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-04-03 02:55:20,547 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/.tmp/family2/7aa1a0c9822f4cdc87eee389a739ec14 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/7aa1a0c9822f4cdc87eee389a739ec14 2023-04-03 02:55:20,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-04-03 02:55:20,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-04-03 02:55:20,552 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 65d190e4c51d6a4a46cb459041dddd1e/family2 of 65d190e4c51d6a4a46cb459041dddd1e into 7aa1a0c9822f4cdc87eee389a739ec14(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:20,552 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:55:20,552 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e., storeName=65d190e4c51d6a4a46cb459041dddd1e/family2, priority=12, startTime=1680490519278; duration=1sec 2023-04-03 02:55:20,553 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-03 02:55:20,553 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a431e83bfacaea3fe93968c2ff1d8e5a/family1 in testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:20,553 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp, totalSize=28.9 K 2023-04-03 02:55:20,553 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501088 2023-04-03 02:55:20,554 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:20,554 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-03 02:55:20,554 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:20,555 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:20,556 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1680490501088 2023-04-03 02:55:20,556 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-04-03 02:55:20,569 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] throttle.PressureAwareThroughputController(145): a431e83bfacaea3fe93968c2ff1d8e5a#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:20,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065] to archive 2023-04-03 02:55:20,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:20,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-5773d36c870a442180b04de860e76703 2023-04-03 02:55:20,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-71a93b5ed2d24e6fa3981dc0d4504b2d 2023-04-03 02:55:20,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-8f09c6e18b424ff181c391767fc1bd6a 2023-04-03 02:55:20,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/family2/testCompactionWithHFileLink=65d190e4c51d6a4a46cb459041dddd1e-980e6943e1e94c7189b57b8d7efbf065 2023-04-03 02:55:20,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-04-03 02:55:20,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-03 02:55:20,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-04-03 02:55:20,657 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:55:20,657 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-04-03 02:55:20,657 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:55:20,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:20,659 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:20,659 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:20,662 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9 empty. 2023-04-03 02:55:20,662 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:20,663 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2 empty. 2023-04-03 02:55:20,663 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:20,663 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762 empty. 2023-04-03 02:55:20,663 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:20,663 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:20,663 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-04-03 02:55:20,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:20,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:21,070 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-04-03 02:55:21,072 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => fb728e0e72df91a59f2a870b06ac0762, NAME => 'ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:21,072 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => dc22fb15c436508e6e6e9e67d65eb6d9, NAME => 'ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:21,072 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6e32a990bbbbb168b8d3ba4aabf4ebb2, NAME => 'ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:21,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:21,382 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family1/68ef84ffcd6e4315a5e0525773ba955f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/68ef84ffcd6e4315a5e0525773ba955f 2023-04-03 02:55:21,386 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 1db75b42190724ca7b89184d5fdf0c8e/family1 of 1db75b42190724ca7b89184d5fdf0c8e into 68ef84ffcd6e4315a5e0525773ba955f(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:21,386 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:21,387 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1680490520085; duration=1sec 2023-04-03 02:55:21,387 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-04-03 02:55:21,387 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-03 02:55:21,387 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family1/f87b48738c3446699aaa7e900f3729e2 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f87b48738c3446699aaa7e900f3729e2 2023-04-03 02:55:21,389 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-03 02:55:21,389 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1912): a431e83bfacaea3fe93968c2ff1d8e5a/family2 is initiating minor compaction (all files) 2023-04-03 02:55:21,389 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a431e83bfacaea3fe93968c2ff1d8e5a/family2 in testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:55:21,389 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp, totalSize=20.4 K 2023-04-03 02:55:21,389 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501088 2023-04-03 02:55:21,390 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:21,390 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:21,391 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in a431e83bfacaea3fe93968c2ff1d8e5a/family1 of a431e83bfacaea3fe93968c2ff1d8e5a into f87b48738c3446699aaa7e900f3729e2(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:21,391 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:21,391 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:21,391 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1680490520482; duration=0sec 2023-04-03 02:55:21,391 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-04-03 02:55:21,391 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-03 02:55:21,393 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-03 02:55:21,393 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1912): 1db75b42190724ca7b89184d5fdf0c8e/family2 is initiating minor compaction (all files) 2023-04-03 02:55:21,394 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1db75b42190724ca7b89184d5fdf0c8e/family2 in testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:55:21,394 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp, totalSize=20.4 K 2023-04-03 02:55:21,394 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490501089 2023-04-03 02:55:21,395 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490502103 2023-04-03 02:55:21,395 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490504025 2023-04-03 02:55:21,396 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490505527 2023-04-03 02:55:21,403 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): a431e83bfacaea3fe93968c2ff1d8e5a#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:21,406 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] throttle.PressureAwareThroughputController(145): 1db75b42190724ca7b89184d5fdf0c8e#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:21,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_] to archive 2023-04-03 02:55:21,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:21,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-8b67d6f0eaba494286ae37d842b80352 2023-04-03 02:55:21,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-f4774bef7f7b45e49dd6fa69652d4563 2023-04-03 02:55:21,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-886ca7a0c9184f7689ba28ae4e7b5e3f 2023-04-03 02:55:21,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-0e65b4f45bf649b1bf9cbefd4a4c8eca 2023-04-03 02:55:21,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family1/f867ef2aa7e34cfdb48edc41ff01761c_SeqId_34_ 2023-04-03 02:55:21,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_] to archive 2023-04-03 02:55:21,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:21,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-f1e39049cadc4a108baccdf6294d044b 2023-04-03 02:55:21,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-ed68c06dbd99442db5aff42ce68e23cd 2023-04-03 02:55:21,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-5af47a6c3b5b4b5498a4e50b6c97b8d0 2023-04-03 02:55:21,460 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d88ebf74ee774be4b1f8c4fe37ee07fe 2023-04-03 02:55:21,460 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family1/030005f1742743109cd2d18d3555172c_SeqId_34_ 2023-04-03 02:55:21,476 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:21,477 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,477 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 6e32a990bbbbb168b8d3ba4aabf4ebb2, disabling compactions & flushes 2023-04-03 02:55:21,477 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,477 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,477 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. after waiting 0 ms 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,478 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing fb728e0e72df91a59f2a870b06ac0762, disabling compactions & flushes 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:21,478 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. after waiting 0 ms 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,478 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing dc22fb15c436508e6e6e9e67d65eb6d9, disabling compactions & flushes 2023-04-03 02:55:21,478 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. after waiting 0 ms 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,478 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,478 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:21,481 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:55:21,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1680490521482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490521482"}]},"ts":"1680490521482"} 2023-04-03 02:55:21,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490521482"}]},"ts":"1680490521482"} 2023-04-03 02:55:21,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490521482"}]},"ts":"1680490521482"} 2023-04-03 02:55:21,484 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-03 02:55:21,485 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:55:21,485 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490521485"}]},"ts":"1680490521485"} 2023-04-03 02:55:21,486 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-04-03 02:55:21,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=dc22fb15c436508e6e6e9e67d65eb6d9, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6e32a990bbbbb168b8d3ba4aabf4ebb2, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fb728e0e72df91a59f2a870b06ac0762, ASSIGN}] 2023-04-03 02:55:21,490 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6e32a990bbbbb168b8d3ba4aabf4ebb2, ASSIGN 2023-04-03 02:55:21,490 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=dc22fb15c436508e6e6e9e67d65eb6d9, ASSIGN 2023-04-03 02:55:21,490 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fb728e0e72df91a59f2a870b06ac0762, ASSIGN 2023-04-03 02:55:21,491 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6e32a990bbbbb168b8d3ba4aabf4ebb2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:21,491 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=dc22fb15c436508e6e6e9e67d65eb6d9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:21,491 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fb728e0e72df91a59f2a870b06ac0762, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:21,643 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=fb728e0e72df91a59f2a870b06ac0762, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,643 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=6e32a990bbbbb168b8d3ba4aabf4ebb2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,643 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=dc22fb15c436508e6e6e9e67d65eb6d9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521643"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490521643"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490521643"}]},"ts":"1680490521643"} 2023-04-03 02:55:21,644 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521643"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490521643"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490521643"}]},"ts":"1680490521643"} 2023-04-03 02:55:21,644 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1680490521643"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490521643"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490521643"}]},"ts":"1680490521643"} 2023-04-03 02:55:21,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure dc22fb15c436508e6e6e9e67d65eb6d9, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:21,645 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure fb728e0e72df91a59f2a870b06ac0762, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:21,646 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure 6e32a990bbbbb168b8d3ba4aabf4ebb2, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:21,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:21,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fb728e0e72df91a59f2a870b06ac0762, NAME => 'ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.', STARTKEY => '2', ENDKEY => ''} 2023-04-03 02:55:21,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,800 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,801 DEBUG [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1 2023-04-03 02:55:21,801 DEBUG [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1 2023-04-03 02:55:21,802 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb728e0e72df91a59f2a870b06ac0762 columnFamilyName family1 2023-04-03 02:55:21,803 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] regionserver.HStore(310): Store=fb728e0e72df91a59f2a870b06ac0762/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,803 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,804 DEBUG [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2 2023-04-03 02:55:21,804 DEBUG [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2 2023-04-03 02:55:21,804 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb728e0e72df91a59f2a870b06ac0762 columnFamilyName family2 2023-04-03 02:55:21,805 INFO [StoreOpener-fb728e0e72df91a59f2a870b06ac0762-1] regionserver.HStore(310): Store=fb728e0e72df91a59f2a870b06ac0762/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:21,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:55:21,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:21,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fb728e0e72df91a59f2a870b06ac0762; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62690948, jitterRate=-0.06583207845687866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:21,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:21,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762., pid=110, masterSystemTime=1680490521796 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:21,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6e32a990bbbbb168b8d3ba4aabf4ebb2, NAME => 'ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.', STARTKEY => '1', ENDKEY => '2'} 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,813 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=fb728e0e72df91a59f2a870b06ac0762, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,813 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521812"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490521812"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490521812"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490521812"}]},"ts":"1680490521812"} 2023-04-03 02:55:21,813 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,815 DEBUG [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 2023-04-03 02:55:21,815 DEBUG [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 2023-04-03 02:55:21,815 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e32a990bbbbb168b8d3ba4aabf4ebb2 columnFamilyName family1 2023-04-03 02:55:21,815 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=108 2023-04-03 02:55:21,815 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure fb728e0e72df91a59f2a870b06ac0762, server=jenkins-hbase4.apache.org,45927,1680490420719 in 169 msec 2023-04-03 02:55:21,816 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] regionserver.HStore(310): Store=6e32a990bbbbb168b8d3ba4aabf4ebb2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,816 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,816 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=fb728e0e72df91a59f2a870b06ac0762, ASSIGN in 326 msec 2023-04-03 02:55:21,817 DEBUG [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2 2023-04-03 02:55:21,817 DEBUG [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2 2023-04-03 02:55:21,818 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e32a990bbbbb168b8d3ba4aabf4ebb2 columnFamilyName family2 2023-04-03 02:55:21,818 INFO [StoreOpener-6e32a990bbbbb168b8d3ba4aabf4ebb2-1] regionserver.HStore(310): Store=6e32a990bbbbb168b8d3ba4aabf4ebb2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,820 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/.tmp/family2/025f09522e854bd9980f9ffcd24146ba as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/025f09522e854bd9980f9ffcd24146ba 2023-04-03 02:55:21,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:21,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:55:21,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:21,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6e32a990bbbbb168b8d3ba4aabf4ebb2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75425814, jitterRate=0.12393221259117126}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:21,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:21,824 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in a431e83bfacaea3fe93968c2ff1d8e5a/family2 of a431e83bfacaea3fe93968c2ff1d8e5a into 025f09522e854bd9980f9ffcd24146ba(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:21,824 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:55:21,824 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a., storeName=a431e83bfacaea3fe93968c2ff1d8e5a/family2, priority=12, startTime=1680490519427; duration=0sec 2023-04-03 02:55:21,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2., pid=111, masterSystemTime=1680490521796 2023-04-03 02:55:21,824 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-03 02:55:21,824 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-03 02:55:21,824 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. because compaction request was cancelled 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-03 02:55:21,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,825 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. because compaction request was cancelled 2023-04-03 02:55:21,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:21,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dc22fb15c436508e6e6e9e67d65eb6d9, NAME => 'ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.', STARTKEY => '', ENDKEY => '1'} 2023-04-03 02:55:21,825 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=6e32a990bbbbb168b8d3ba4aabf4ebb2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,825 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1680490521825"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490521825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490521825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490521825"}]},"ts":"1680490521825"} 2023-04-03 02:55:21,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:21,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,826 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,827 DEBUG [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1 2023-04-03 02:55:21,827 DEBUG [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1 2023-04-03 02:55:21,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-04-03 02:55:21,828 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure 6e32a990bbbbb168b8d3ba4aabf4ebb2, server=jenkins-hbase4.apache.org,45927,1680490420719 in 180 msec 2023-04-03 02:55:21,828 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc22fb15c436508e6e6e9e67d65eb6d9 columnFamilyName family1 2023-04-03 02:55:21,828 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6e32a990bbbbb168b8d3ba4aabf4ebb2, ASSIGN in 339 msec 2023-04-03 02:55:21,828 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] regionserver.HStore(310): Store=dc22fb15c436508e6e6e9e67d65eb6d9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,829 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,830 DEBUG [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2 2023-04-03 02:55:21,830 DEBUG [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2 2023-04-03 02:55:21,830 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc22fb15c436508e6e6e9e67d65eb6d9 columnFamilyName family2 2023-04-03 02:55:21,830 INFO [StoreOpener-dc22fb15c436508e6e6e9e67d65eb6d9-1] regionserver.HStore(310): Store=dc22fb15c436508e6e6e9e67d65eb6d9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:21,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:21,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:55:21,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:21,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dc22fb15c436508e6e6e9e67d65eb6d9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62158143, jitterRate=-0.07377149164676666}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:21,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:21,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9., pid=109, masterSystemTime=1680490521796 2023-04-03 02:55:21,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:21,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=dc22fb15c436508e6e6e9e67d65eb6d9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:21,836 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1680490521836"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490521836"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490521836"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490521836"}]},"ts":"1680490521836"} 2023-04-03 02:55:21,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-04-03 02:55:21,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure dc22fb15c436508e6e6e9e67d65eb6d9, server=jenkins-hbase4.apache.org,45927,1680490420719 in 192 msec 2023-04-03 02:55:21,839 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-04-03 02:55:21,839 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=dc22fb15c436508e6e6e9e67d65eb6d9, ASSIGN in 349 msec 2023-04-03 02:55:21,840 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:55:21,840 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490521840"}]},"ts":"1680490521840"} 2023-04-03 02:55:21,841 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-04-03 02:55:21,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef] to archive 2023-04-03 02:55:21,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:21,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-85917e242a1b4e6f8444acd5c29d594f 2023-04-03 02:55:21,843 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:55:21,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-9acebd5f73144c5cb35cd6bdad143f6f 2023-04-03 02:55:21,844 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1890 sec 2023-04-03 02:55:21,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-2ed38c058b6943c89bebded073e16732 2023-04-03 02:55:21,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/family2/testCompactionWithHFileLink=a431e83bfacaea3fe93968c2ff1d8e5a-d99c992a13a94d3497fd4958f8b237ef 2023-04-03 02:55:22,624 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/.tmp/family2/b7adff088a4d46f8a5a933a63189894e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/b7adff088a4d46f8a5a933a63189894e 2023-04-03 02:55:22,627 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1db75b42190724ca7b89184d5fdf0c8e/family2 of 1db75b42190724ca7b89184d5fdf0c8e into b7adff088a4d46f8a5a933a63189894e(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:22,628 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:55:22,628 INFO [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e., storeName=1db75b42190724ca7b89184d5fdf0c8e/family2, priority=12, startTime=1680490519368; duration=1sec 2023-04-03 02:55:22,628 DEBUG [RS:0;jenkins-hbase4:45927-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:22,646 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19] to archive 2023-04-03 02:55:22,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:22,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-7e893c05310a4d6d911a5f1089d47271 2023-04-03 02:55:22,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-47e8e37d6b8d4aed96bcc79fe101013a 2023-04-03 02:55:22,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-fb825f81bafd47c0870f9fb4db1c1aaa 2023-04-03 02:55:22,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/family2/testCompactionWithHFileLink=1db75b42190724ca7b89184d5fdf0c8e-d647b4cce0e84a5d94ae6e0188f6bc19 2023-04-03 02:55:22,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-03 02:55:22,762 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-04-03 02:55:22,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:22,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:22,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:22,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:22,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-03 02:55:22,772 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-03 02:55:22,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:22,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-03 02:55:22,772 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-03 02:55:22,773 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:22,773 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:22,774 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:22,774 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:22,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:22,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:22,774 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:22,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:22,774 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:22,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:22,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:22,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:22,775 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-03 02:55:22,775 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-03 02:55:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-03 02:55:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:22,775 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-03 02:55:22,776 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:22,776 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. started... 2023-04-03 02:55:22,776 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:22,776 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. started... 2023-04-03 02:55:22,777 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:22,778 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dc22fb15c436508e6e6e9e67d65eb6d9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:22,776 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:22,778 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:22,778 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6e32a990bbbbb168b8d3ba4aabf4ebb2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:22,778 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. started... 2023-04-03 02:55:22,778 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fb728e0e72df91a59f2a870b06ac0762 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:22,798 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304031a3baee927ae49c0942de80d7d7cf7c7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304031a3baee927ae49c0942de80d7d7cf7c7 2023-04-03 02:55:23,197 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403c7aa25be200640feba5a5848e8eebda1 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403c7aa25be200640feba5a5848e8eebda1 2023-04-03 02:55:23,198 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403268c1edf105745e59f1ec5e80da07dad to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230403268c1edf105745e59f1ec5e80da07dad 2023-04-03 02:55:23,206 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/397bbdce7aab40bab50bd1f8d59c61b7 2023-04-03 02:55:23,602 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/f1111551d9f143fdbee2fabfff9c9460 2023-04-03 02:55:23,602 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/3f73c11551c54ac980f4573b969344f8 2023-04-03 02:55:23,616 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304032eec13a8237e43b9a145bd3e50eda5b0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202304032eec13a8237e43b9a145bd3e50eda5b0 2023-04-03 02:55:23,617 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403bdbce69c6b2f42f7b6d53654a3d9f58e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230403bdbce69c6b2f42f7b6d53654a3d9f58e 2023-04-03 02:55:24,017 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304037cdd20c9098241bc869083e8de73de2f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202304037cdd20c9098241bc869083e8de73de2f 2023-04-03 02:55:24,024 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/b89a0dd00cd34b458dcb61dcebfa415a 2023-04-03 02:55:24,025 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/ae44987f45f24d03af025b21477298be 2023-04-03 02:55:24,030 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/8cb90ffbc35b4cf197cd1342912e0265 2023-04-03 02:55:24,032 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/397bbdce7aab40bab50bd1f8d59c61b7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7 2023-04-03 02:55:24,034 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/3f73c11551c54ac980f4573b969344f8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8 2023-04-03 02:55:24,043 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/f1111551d9f143fdbee2fabfff9c9460 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460 2023-04-03 02:55:24,045 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:24,046 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/b89a0dd00cd34b458dcb61dcebfa415a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a 2023-04-03 02:55:24,046 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:24,047 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/ae44987f45f24d03af025b21477298be as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be 2023-04-03 02:55:24,048 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460, entries=4, sequenceid=5, filesize=5.4 K 2023-04-03 02:55:24,058 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/8cb90ffbc35b4cf197cd1342912e0265 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265 2023-04-03 02:55:24,061 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:24,061 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:24,063 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:24,068 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fb728e0e72df91a59f2a870b06ac0762 in 1290ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:24,068 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for dc22fb15c436508e6e6e9e67d65eb6d9 in 1291ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:24,068 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-04-03 02:55:24,068 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6e32a990bbbbb168b8d3ba4aabf4ebb2 in 1290ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:24,068 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-04-03 02:55:24,069 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:24,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:24,068 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-04-03 02:55:24,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:24,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:24,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:24,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-03 02:55:24,070 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-03 02:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,072 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:24,076 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:24,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:24,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-03 02:55:24,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@152f5112[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:24,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-03 02:55:24,078 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,081 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,081 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-03 02:55:24,081 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-03 02:55:24,081 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:24,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,081 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:24,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-03 02:55:24,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:24,082 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,082 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-03 02:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:24,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:24,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:24,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-03 02:55:24,086 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-03 02:55:24,086 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:24,086 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:24,086 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-03 02:55:24,086 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:24,088 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,088 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:24,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:24,088 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:24,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:24,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:24,088 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:24,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:24,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,090 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:24,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:24,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:24,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:24,092 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-04-03 02:55:24,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:24,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-03 02:55:24,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:24,092 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:24,092 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,093 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,093 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:24,093 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,093 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:24,093 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:24,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:24,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:24,193 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:24,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:24,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:24,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:24,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:24,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:24,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:24,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-03 02:55:24,201 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-03 02:55:24,201 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:24,201 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-03 02:55:24,201 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-03 02:55:24,202 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,202 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:24,204 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:24,204 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:24,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:24,204 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:24,204 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:24,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:24,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:24,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:24,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:24,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-03 02:55:24,206 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-03 02:55:24,206 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-03 02:55:24,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-03 02:55:24,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:24,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-03 02:55:24,206 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:24,206 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. started... 2023-04-03 02:55:24,206 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dc22fb15c436508e6e6e9e67d65eb6d9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:24,208 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:24,208 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:24,208 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:24,208 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:24,208 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. started... 2023-04-03 02:55:24,208 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. started... 2023-04-03 02:55:24,209 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fb728e0e72df91a59f2a870b06ac0762 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:24,209 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6e32a990bbbbb168b8d3ba4aabf4ebb2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:24,246 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040309a3363fb2164aaba13663c98966b33b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023040309a3363fb2164aaba13663c98966b33b 2023-04-03 02:55:24,258 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/eb11e83b6d60406da6fc4f7b668d2d0f 2023-04-03 02:55:24,269 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403a6d10a1da90d4698b80e9388068b831a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230403a6d10a1da90d4698b80e9388068b831a 2023-04-03 02:55:24,638 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304037074c2073eb24bd79099df6373efa151 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304037074c2073eb24bd79099df6373efa151 2023-04-03 02:55:24,638 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304034e84f7966b8a4a1594e1b5610eb9e893 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202304034e84f7966b8a4a1594e1b5610eb9e893 2023-04-03 02:55:24,649 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/0565564133e04ce18b0c3dcf62c391bc 2023-04-03 02:55:24,670 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403e2d0ab7d34074995baf5dbab421fee1a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230403e2d0ab7d34074995baf5dbab421fee1a 2023-04-03 02:55:24,675 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/01e82140e7bb4fbfa46bebe602aa317a 2023-04-03 02:55:24,677 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/07a23bd5bf2f476ab4f9789d00456e49 2023-04-03 02:55:24,681 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/eb11e83b6d60406da6fc4f7b668d2d0f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f 2023-04-03 02:55:24,681 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/0565564133e04ce18b0c3dcf62c391bc as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc 2023-04-03 02:55:24,685 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:24,685 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:24,686 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/01e82140e7bb4fbfa46bebe602aa317a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a 2023-04-03 02:55:24,686 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/07a23bd5bf2f476ab4f9789d00456e49 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49 2023-04-03 02:55:24,690 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:24,690 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:24,691 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fb728e0e72df91a59f2a870b06ac0762 in 482ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:24,691 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6e32a990bbbbb168b8d3ba4aabf4ebb2 in 482ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:24,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:24,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:24,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:24,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:24,691 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:24,691 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:25,046 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/5cb91845cae14c67b7089931985f8956 2023-04-03 02:55:25,461 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403e37a0bced2264a319549dadffec56421 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230403e37a0bced2264a319549dadffec56421 2023-04-03 02:55:25,873 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/019354d69c494be7a62f02d4d2114d83 2023-04-03 02:55:25,878 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/5cb91845cae14c67b7089931985f8956 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956 2023-04-03 02:55:25,882 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956, entries=4, sequenceid=9, filesize=5.4 K 2023-04-03 02:55:25,882 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/019354d69c494be7a62f02d4d2114d83 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83 2023-04-03 02:55:25,886 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:25,887 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for dc22fb15c436508e6e6e9e67d65eb6d9 in 1681ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:25,887 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:25,887 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-03 02:55:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-03 02:55:25,889 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:25,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:25,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:25,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-03 02:55:25,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@122656a8[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:25,891 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-03 02:55:25,891 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,892 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-03 02:55:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-03 02:55:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:25,892 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,892 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:25,897 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-03 02:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:25,897 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-03 02:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-03 02:55:25,899 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-03 02:55:25,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:25,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:25,899 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-03 02:55:25,900 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:25,901 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:25,901 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,901 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:25,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:25,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:25,902 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,903 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:25,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:25,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:25,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:25,905 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:25,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:25,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:25,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:25,905 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:25,906 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:25,906 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:25,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:25,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:26,006 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:26,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:26,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:26,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:26,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:26,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:26,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:26,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-03 02:55:26,020 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-03 02:55:26,020 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:26,020 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-03 02:55:26,020 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-03 02:55:26,021 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:26,021 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:26,023 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:26,023 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:26,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:26,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:26,023 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:26,023 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:26,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:26,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:26,024 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-03 02:55:26,024 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-03 02:55:26,024 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-03 02:55:26,024 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-03 02:55:26,024 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:26,025 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-03 02:55:26,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:26,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:26,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. started... 2023-04-03 02:55:26,025 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. started... 2023-04-03 02:55:26,025 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dc22fb15c436508e6e6e9e67d65eb6d9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:26,025 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6e32a990bbbbb168b8d3ba4aabf4ebb2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:26,026 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:26,026 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:26,026 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. started... 2023-04-03 02:55:26,026 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fb728e0e72df91a59f2a870b06ac0762 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:26,059 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403d0e9a37320c34ddab0e7524144ed4c91 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230403d0e9a37320c34ddab0e7524144ed4c91 2023-04-03 02:55:26,064 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304038ad64e720ebb41f8aaaa48956af5f8e7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304038ad64e720ebb41f8aaaa48956af5f8e7 2023-04-03 02:55:26,072 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/db884996dd954efe9b1144481e3e7dd7 2023-04-03 02:55:26,112 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/8c868568e8c54dbaaad38572bae2baba 2023-04-03 02:55:26,125 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403800f4ab92ba54995946e9bc4e5d63936 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230403800f4ab92ba54995946e9bc4e5d63936 2023-04-03 02:55:26,133 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/4cb00c12e6144d9aa340d51781acb91d 2023-04-03 02:55:26,138 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/8c868568e8c54dbaaad38572bae2baba as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba 2023-04-03 02:55:26,144 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:26,145 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/4cb00c12e6144d9aa340d51781acb91d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d 2023-04-03 02:55:26,150 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:26,151 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fb728e0e72df91a59f2a870b06ac0762 in 125ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:26,151 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:26,151 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:26,151 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:26,454 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304033bb51c63836b4dd1a60b1051ea8a35d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202304033bb51c63836b4dd1a60b1051ea8a35d0 2023-04-03 02:55:26,518 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403ca9990df1fe045b59c5d0b2638234738 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230403ca9990df1fe045b59c5d0b2638234738 2023-04-03 02:55:26,523 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/a3d03ced85414edebcec365b54dd825b 2023-04-03 02:55:26,527 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/db884996dd954efe9b1144481e3e7dd7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7 2023-04-03 02:55:26,531 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:26,531 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/a3d03ced85414edebcec365b54dd825b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b 2023-04-03 02:55:26,535 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:26,536 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for dc22fb15c436508e6e6e9e67d65eb6d9 in 511ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:26,536 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:26,536 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:26,536 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:26,861 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/ff2b624db80540f6900707d03a72602a 2023-04-03 02:55:27,275 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304038b7cfc4769574e81af90896ed5d5834c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202304038b7cfc4769574e81af90896ed5d5834c 2023-04-03 02:55:27,301 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:27,680 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/281b2190050a41a1920780d19d04dcc4 2023-04-03 02:55:27,685 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/ff2b624db80540f6900707d03a72602a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a 2023-04-03 02:55:27,691 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a, entries=4, sequenceid=13, filesize=5.4 K 2023-04-03 02:55:27,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/281b2190050a41a1920780d19d04dcc4 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4 2023-04-03 02:55:27,695 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:27,696 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6e32a990bbbbb168b8d3ba4aabf4ebb2 in 1671ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:27,696 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:27,697 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-03 02:55:27,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-03 02:55:27,698 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,698 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:27,699 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,699 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:27,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-03 02:55:27,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c1587[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:27,700 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-03 02:55:27,700 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,701 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-03 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-03 02:55:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:27,702 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,702 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:27,703 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-03 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:27,703 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-03 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:27,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:27,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:27,705 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-03 02:55:27,705 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-03 02:55:27,705 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:27,705 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:27,705 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-03 02:55:27,705 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:27,707 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,707 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:27,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,707 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:27,708 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:27,709 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:27,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,711 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,711 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:27,711 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:27,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:27,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-03 02:55:27,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:27,711 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:27,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:27,711 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:27,711 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:27,712 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-04-03 02:55:27,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:27,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:27,812 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:27,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:27,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:27,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:27,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:27,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:27,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:27,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-03 02:55:27,819 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-03 02:55:27,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:27,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-03 02:55:27,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-03 02:55:27,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:27,821 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:27,821 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:27,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:27,821 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:27,821 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:27,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:27,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:27,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:27,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:27,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-03 02:55:27,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-03 02:55:27,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-03 02:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-03 02:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:27,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-03 02:55:27,822 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:27,822 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. started... 2023-04-03 02:55:27,822 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dc22fb15c436508e6e6e9e67d65eb6d9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:27,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:27,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. started... 2023-04-03 02:55:27,823 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:27,823 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:27,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:27,823 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6e32a990bbbbb168b8d3ba4aabf4ebb2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:27,823 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. started... 2023-04-03 02:55:27,823 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fb728e0e72df91a59f2a870b06ac0762 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:27,849 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304039bf00055da604b0f8279d31d94b03606 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304039bf00055da604b0f8279d31d94b03606 2023-04-03 02:55:27,859 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/abb060051b4e4a9f9ce94838c8ac5e7d 2023-04-03 02:55:27,873 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304033482f40f824a4baba0eceb81c75a1319 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202304033482f40f824a4baba0eceb81c75a1319 2023-04-03 02:55:27,888 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/1ec9b97e7c5b436b892d8c4d316e65a4 2023-04-03 02:55:27,892 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/abb060051b4e4a9f9ce94838c8ac5e7d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d 2023-04-03 02:55:27,898 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:27,899 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/1ec9b97e7c5b436b892d8c4d316e65a4 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4 2023-04-03 02:55:27,907 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:27,908 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for dc22fb15c436508e6e6e9e67d65eb6d9 in 86ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:27,908 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:27,908 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:27,908 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:28,246 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403846a519e97db4ad7a421477260397960 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403846a519e97db4ad7a421477260397960 2023-04-03 02:55:28,250 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304035690a42d91234398b7e47bb1264d9ecf to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304035690a42d91234398b7e47bb1264d9ecf 2023-04-03 02:55:28,252 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/eb5b5d6f02c348fcb298843ae4a46e14 2023-04-03 02:55:28,255 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/aed8242ccffc486b88051f8c0956cc43 2023-04-03 02:55:28,682 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403139f98337b0a429ab0e726c0089c7174 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230403139f98337b0a429ab0e726c0089c7174 2023-04-03 02:55:28,691 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403555933d169754b40b02e011b03d756b3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230403555933d169754b40b02e011b03d756b3 2023-04-03 02:55:28,697 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/7ec851cf529e4b98ae0b57855b36b2ab 2023-04-03 02:55:28,703 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/aed8242ccffc486b88051f8c0956cc43 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43 2023-04-03 02:55:28,708 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:28,709 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/7ec851cf529e4b98ae0b57855b36b2ab as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab 2023-04-03 02:55:28,714 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:28,715 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fb728e0e72df91a59f2a870b06ac0762 in 892ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:28,715 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:28,715 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:28,715 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:29,090 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/d262825699ef45439aad2ee7626ef3fe 2023-04-03 02:55:29,095 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/eb5b5d6f02c348fcb298843ae4a46e14 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14 2023-04-03 02:55:29,098 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14, entries=4, sequenceid=17, filesize=5.4 K 2023-04-03 02:55:29,099 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/d262825699ef45439aad2ee7626ef3fe as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe 2023-04-03 02:55:29,103 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:29,104 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6e32a990bbbbb168b8d3ba4aabf4ebb2 in 1281ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:29,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:29,104 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:29,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:29,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:29,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-03 02:55:29,104 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-03 02:55:29,107 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,107 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:29,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:29,107 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,107 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:29,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:29,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:29,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-03 02:55:29,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ece12d8[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:29,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-03 02:55:29,109 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,110 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-03 02:55:29,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-03 02:55:29,111 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:29,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,111 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:29,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-03 02:55:29,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:29,112 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,112 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-03 02:55:29,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:29,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:29,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:29,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:29,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:29,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:29,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-03 02:55:29,116 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-03 02:55:29,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:29,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:29,116 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-03 02:55:29,116 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:29,117 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,117 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:29,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:29,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,117 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:29,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,118 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,125 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:29,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:29,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:29,127 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,128 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:29,128 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,128 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:29,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-03 02:55:29,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:29,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:29,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:29,128 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:29,128 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:29,228 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:29,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:29,250 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:29,265 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:29,287 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:29,294 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:29,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:29,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:29,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-03 02:55:29,346 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-03 02:55:29,346 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:29,346 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-03 02:55:29,346 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-03 02:55:29,347 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,347 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:29,349 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:29,349 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:29,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:29,349 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:29,349 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:29,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:29,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:29,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:29,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:29,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-03 02:55:29,351 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-03 02:55:29,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-03 02:55:29,351 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-03 02:55:29,351 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:29,351 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-03 02:55:29,352 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:29,352 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:29,352 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. started... 2023-04-03 02:55:29,352 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. started... 2023-04-03 02:55:29,352 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6e32a990bbbbb168b8d3ba4aabf4ebb2 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:29,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:29,352 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dc22fb15c436508e6e6e9e67d65eb6d9 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:29,353 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:29,353 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. started... 2023-04-03 02:55:29,353 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fb728e0e72df91a59f2a870b06ac0762 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:29,804 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/ece775497b2143a4aa47f5fa9840ec45 2023-04-03 02:55:29,808 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ece775497b2143a4aa47f5fa9840ec45 2023-04-03 02:55:29,825 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/f94d5f393f604b0abb8de6f6862ebe12 2023-04-03 02:55:29,826 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/9c332efdf32e4e138d828911cbcb7f55 2023-04-03 02:55:29,830 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f94d5f393f604b0abb8de6f6862ebe12 2023-04-03 02:55:29,831 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9c332efdf32e4e138d828911cbcb7f55 2023-04-03 02:55:29,847 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/428c1ae45faf4f0e895b7714b5b57abd 2023-04-03 02:55:29,850 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 428c1ae45faf4f0e895b7714b5b57abd 2023-04-03 02:55:29,851 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/9c332efdf32e4e138d828911cbcb7f55 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55 2023-04-03 02:55:29,854 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9c332efdf32e4e138d828911cbcb7f55 2023-04-03 02:55:29,854 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55, entries=3, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:29,856 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/428c1ae45faf4f0e895b7714b5b57abd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd 2023-04-03 02:55:29,860 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 428c1ae45faf4f0e895b7714b5b57abd 2023-04-03 02:55:29,860 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:29,861 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fb728e0e72df91a59f2a870b06ac0762 in 508ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:29,861 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:29,861 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:29,861 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:30,222 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/c8b247376cf8401cb6f49c860b4791c7 2023-04-03 02:55:30,226 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8b247376cf8401cb6f49c860b4791c7 2023-04-03 02:55:30,227 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/ece775497b2143a4aa47f5fa9840ec45 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45 2023-04-03 02:55:30,230 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ece775497b2143a4aa47f5fa9840ec45 2023-04-03 02:55:30,230 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45, entries=3, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:30,230 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/c8b247376cf8401cb6f49c860b4791c7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7 2023-04-03 02:55:30,234 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8b247376cf8401cb6f49c860b4791c7 2023-04-03 02:55:30,234 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:30,235 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6e32a990bbbbb168b8d3ba4aabf4ebb2 in 883ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:30,235 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:30,235 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:30,235 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:30,248 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/d6e247c17c2c4e0a94cf69b362d83fea 2023-04-03 02:55:30,251 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6e247c17c2c4e0a94cf69b362d83fea 2023-04-03 02:55:30,252 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/f94d5f393f604b0abb8de6f6862ebe12 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12 2023-04-03 02:55:30,256 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f94d5f393f604b0abb8de6f6862ebe12 2023-04-03 02:55:30,257 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12, entries=3, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:30,257 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/d6e247c17c2c4e0a94cf69b362d83fea as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea 2023-04-03 02:55:30,261 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d6e247c17c2c4e0a94cf69b362d83fea 2023-04-03 02:55:30,261 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:30,262 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for dc22fb15c436508e6e6e9e67d65eb6d9 in 910ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:30,262 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:30,262 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-03 02:55:30,262 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-03 02:55:30,263 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,263 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:30,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:30,264 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,264 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:30,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:30,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:30,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:30,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-03 02:55:30,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@618665d2[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:30,265 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-03 02:55:30,265 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,266 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,266 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-03 02:55:30,266 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-03 02:55:30,266 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:30,266 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,266 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:30,267 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-03 02:55:30,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:30,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:30,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:30,267 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-03 02:55:30,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:30,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:30,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:30,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:30,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-03 02:55:30,270 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-03 02:55:30,270 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:30,270 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:30,270 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-03 02:55:30,270 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:30,275 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,275 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:30,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,275 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:30,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:30,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:30,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:30,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,277 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:30,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-03 02:55:30,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:30,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-03 02:55:30,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:30,280 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:30,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:30,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:30,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:30,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:30,280 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,281 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-03 02:55:30,281 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-03 02:55:30,380 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:30,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:30,382 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:30,382 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): dc22fb15c436508e6e6e9e67d65eb6d9/family1 is initiating major compaction (all files) 2023-04-03 02:55:30,382 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:30,382 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:30,383 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of dc22fb15c436508e6e6e9e67d65eb6d9/family1 in ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:30,383 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp, totalSize=26.6 K 2023-04-03 02:55:30,384 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 397bbdce7aab40bab50bd1f8d59c61b7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522768 2023-04-03 02:55:30,384 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 5cb91845cae14c67b7089931985f8956, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:30,385 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting db884996dd954efe9b1144481e3e7dd7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:30,385 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting abb060051b4e4a9f9ce94838c8ac5e7d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527814 2023-04-03 02:55:30,386 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting f94d5f393f604b0abb8de6f6862ebe12, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:30,396 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): dc22fb15c436508e6e6e9e67d65eb6d9#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:30,806 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023040375fed355bc3a44b3bf191912f19d76a4_del 2023-04-03 02:55:30,806 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040375fed355bc3a44b3bf191912f19d76a4_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023040375fed355bc3a44b3bf191912f19d76a4_del 2023-04-03 02:55:31,225 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 066772cc6948447a834c019e75aed467 2023-04-03 02:55:31,226 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family1/066772cc6948447a834c019e75aed467 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/066772cc6948447a834c019e75aed467 2023-04-03 02:55:31,229 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 066772cc6948447a834c019e75aed467 2023-04-03 02:55:31,230 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dc22fb15c436508e6e6e9e67d65eb6d9/family1 of dc22fb15c436508e6e6e9e67d65eb6d9 into 066772cc6948447a834c019e75aed467(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:31,231 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:31,231 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:31,231 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): dc22fb15c436508e6e6e9e67d65eb6d9/family2 is initiating major compaction (all files) 2023-04-03 02:55:31,231 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:31,231 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:31,231 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of dc22fb15c436508e6e6e9e67d65eb6d9/family2 in ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:55:31,231 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp, totalSize=25.3 K 2023-04-03 02:55:31,232 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting b89a0dd00cd34b458dcb61dcebfa415a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522768 2023-04-03 02:55:31,232 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 019354d69c494be7a62f02d4d2114d83, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:31,232 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting a3d03ced85414edebcec365b54dd825b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:31,233 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 1ec9b97e7c5b436b892d8c4d316e65a4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527814 2023-04-03 02:55:31,233 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d6e247c17c2c4e0a94cf69b362d83fea, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:31,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12] to archive 2023-04-03 02:55:31,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:31,243 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/397bbdce7aab40bab50bd1f8d59c61b7 2023-04-03 02:55:31,243 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): dc22fb15c436508e6e6e9e67d65eb6d9#family2#compaction#150 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:31,244 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/5cb91845cae14c67b7089931985f8956 2023-04-03 02:55:31,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/db884996dd954efe9b1144481e3e7dd7 2023-04-03 02:55:31,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/abb060051b4e4a9f9ce94838c8ac5e7d 2023-04-03 02:55:31,249 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/f94d5f393f604b0abb8de6f6862ebe12 2023-04-03 02:55:31,261 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023040396d229ee16c2418aaaaa648c94e81b7c_del 2023-04-03 02:55:31,261 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040396d229ee16c2418aaaaa648c94e81b7c_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023040396d229ee16c2418aaaaa648c94e81b7c_del 2023-04-03 02:55:32,079 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 209b0449225d44799af0c4f221d7e33b 2023-04-03 02:55:32,079 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/.tmp/family2/209b0449225d44799af0c4f221d7e33b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/209b0449225d44799af0c4f221d7e33b 2023-04-03 02:55:32,083 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 209b0449225d44799af0c4f221d7e33b 2023-04-03 02:55:32,083 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dc22fb15c436508e6e6e9e67d65eb6d9/family2 of dc22fb15c436508e6e6e9e67d65eb6d9 into 209b0449225d44799af0c4f221d7e33b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:32,084 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:32,084 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:32,084 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 is initiating major compaction (all files) 2023-04-03 02:55:32,084 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:32,084 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:32,085 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 in ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:32,085 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp, totalSize=26.6 K 2023-04-03 02:55:32,085 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting f1111551d9f143fdbee2fabfff9c9460, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522769 2023-04-03 02:55:32,086 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 0565564133e04ce18b0c3dcf62c391bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:32,086 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting ff2b624db80540f6900707d03a72602a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:32,086 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting eb5b5d6f02c348fcb298843ae4a46e14, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527814 2023-04-03 02:55:32,087 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting ece775497b2143a4aa47f5fa9840ec45, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:32,100 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 6e32a990bbbbb168b8d3ba4aabf4ebb2#family1#compaction#151 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:32,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea] to archive 2023-04-03 02:55:32,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:32,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/b89a0dd00cd34b458dcb61dcebfa415a 2023-04-03 02:55:32,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/019354d69c494be7a62f02d4d2114d83 2023-04-03 02:55:32,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/a3d03ced85414edebcec365b54dd825b 2023-04-03 02:55:32,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/1ec9b97e7c5b436b892d8c4d316e65a4 2023-04-03 02:55:32,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family2/d6e247c17c2c4e0a94cf69b362d83fea 2023-04-03 02:55:32,508 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230403e8a6e20622374e039ea919ba6d7c6c40_del 2023-04-03 02:55:32,508 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403e8a6e20622374e039ea919ba6d7c6c40_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403e8a6e20622374e039ea919ba6d7c6c40_del 2023-04-03 02:55:32,521 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7cce74e856d4b75873d7c57324e22fb 2023-04-03 02:55:32,521 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family1/f7cce74e856d4b75873d7c57324e22fb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f7cce74e856d4b75873d7c57324e22fb 2023-04-03 02:55:32,525 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7cce74e856d4b75873d7c57324e22fb 2023-04-03 02:55:32,526 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 of 6e32a990bbbbb168b8d3ba4aabf4ebb2 into f7cce74e856d4b75873d7c57324e22fb(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:32,526 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:32,526 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:32,526 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 6e32a990bbbbb168b8d3ba4aabf4ebb2/family2 is initiating major compaction (all files) 2023-04-03 02:55:32,526 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:32,527 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:32,527 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 6e32a990bbbbb168b8d3ba4aabf4ebb2/family2 in ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:55:32,527 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp, totalSize=25.3 K 2023-04-03 02:55:32,529 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 8cb90ffbc35b4cf197cd1342912e0265, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522769 2023-04-03 02:55:32,529 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 07a23bd5bf2f476ab4f9789d00456e49, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:32,530 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 281b2190050a41a1920780d19d04dcc4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:32,530 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d262825699ef45439aad2ee7626ef3fe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527814 2023-04-03 02:55:32,530 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting c8b247376cf8401cb6f49c860b4791c7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:32,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45] to archive 2023-04-03 02:55:32,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:32,544 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/f1111551d9f143fdbee2fabfff9c9460 2023-04-03 02:55:32,545 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/0565564133e04ce18b0c3dcf62c391bc 2023-04-03 02:55:32,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ff2b624db80540f6900707d03a72602a 2023-04-03 02:55:32,547 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 6e32a990bbbbb168b8d3ba4aabf4ebb2#family2#compaction#152 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:32,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/eb5b5d6f02c348fcb298843ae4a46e14 2023-04-03 02:55:32,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/ece775497b2143a4aa47f5fa9840ec45 2023-04-03 02:55:32,979 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202304035ace7a7c9aae408181a346fc7a88213e_del 2023-04-03 02:55:32,980 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304035ace7a7c9aae408181a346fc7a88213e_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202304035ace7a7c9aae408181a346fc7a88213e_del 2023-04-03 02:55:32,997 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22ef23df0ea3467397c53e4b77d8db1f 2023-04-03 02:55:32,997 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/.tmp/family2/22ef23df0ea3467397c53e4b77d8db1f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/22ef23df0ea3467397c53e4b77d8db1f 2023-04-03 02:55:33,003 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 22ef23df0ea3467397c53e4b77d8db1f 2023-04-03 02:55:33,004 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6e32a990bbbbb168b8d3ba4aabf4ebb2/family2 of 6e32a990bbbbb168b8d3ba4aabf4ebb2 into 22ef23df0ea3467397c53e4b77d8db1f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:33,004 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:33,004 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:33,004 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): fb728e0e72df91a59f2a870b06ac0762/family1 is initiating major compaction (all files) 2023-04-03 02:55:33,005 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:33,005 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:33,005 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of fb728e0e72df91a59f2a870b06ac0762/family1 in ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:33,005 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp, totalSize=26.6 K 2023-04-03 02:55:33,006 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 3f73c11551c54ac980f4573b969344f8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522769 2023-04-03 02:55:33,007 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting eb11e83b6d60406da6fc4f7b668d2d0f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:33,007 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 8c868568e8c54dbaaad38572bae2baba, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:33,007 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting aed8242ccffc486b88051f8c0956cc43, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527815 2023-04-03 02:55:33,008 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 9c332efdf32e4e138d828911cbcb7f55, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:33,020 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): fb728e0e72df91a59f2a870b06ac0762#family1#compaction#153 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:33,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7] to archive 2023-04-03 02:55:33,042 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:33,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/8cb90ffbc35b4cf197cd1342912e0265 2023-04-03 02:55:33,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/07a23bd5bf2f476ab4f9789d00456e49 2023-04-03 02:55:33,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/281b2190050a41a1920780d19d04dcc4 2023-04-03 02:55:33,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/d262825699ef45439aad2ee7626ef3fe 2023-04-03 02:55:33,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family2/c8b247376cf8401cb6f49c860b4791c7 2023-04-03 02:55:33,434 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403a2d99e4c037c45028f189f48d51683d2_del 2023-04-03 02:55:33,434 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403a2d99e4c037c45028f189f48d51683d2_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230403a2d99e4c037c45028f189f48d51683d2_del 2023-04-03 02:55:34,249 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bef45f4f39eb4fc19dce7f8d0f10bf6f 2023-04-03 02:55:34,249 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family1/bef45f4f39eb4fc19dce7f8d0f10bf6f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/bef45f4f39eb4fc19dce7f8d0f10bf6f 2023-04-03 02:55:34,253 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bef45f4f39eb4fc19dce7f8d0f10bf6f 2023-04-03 02:55:34,255 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fb728e0e72df91a59f2a870b06ac0762/family1 of fb728e0e72df91a59f2a870b06ac0762 into bef45f4f39eb4fc19dce7f8d0f10bf6f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:55:34,255 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:34,255 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:34,255 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): fb728e0e72df91a59f2a870b06ac0762/family2 is initiating major compaction (all files) 2023-04-03 02:55:34,255 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:34,255 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:34,257 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of fb728e0e72df91a59f2a870b06ac0762/family2 in ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:55:34,257 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp, totalSize=25.3 K 2023-04-03 02:55:34,258 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting ae44987f45f24d03af025b21477298be, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490522769 2023-04-03 02:55:34,258 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 01e82140e7bb4fbfa46bebe602aa317a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490524195 2023-04-03 02:55:34,258 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 4cb00c12e6144d9aa340d51781acb91d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490526008 2023-04-03 02:55:34,259 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 7ec851cf529e4b98ae0b57855b36b2ab, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490527815 2023-04-03 02:55:34,260 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 428c1ae45faf4f0e895b7714b5b57abd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:34,273 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): fb728e0e72df91a59f2a870b06ac0762#family2#compaction#154 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:34,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55] to archive 2023-04-03 02:55:34,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:34,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/3f73c11551c54ac980f4573b969344f8 2023-04-03 02:55:34,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/eb11e83b6d60406da6fc4f7b668d2d0f 2023-04-03 02:55:34,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/8c868568e8c54dbaaad38572bae2baba 2023-04-03 02:55:34,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/aed8242ccffc486b88051f8c0956cc43 2023-04-03 02:55:34,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/9c332efdf32e4e138d828911cbcb7f55 2023-04-03 02:55:34,690 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023040368f5dd7d44694a1cbb1af234e43effc4_del 2023-04-03 02:55:34,690 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040368f5dd7d44694a1cbb1af234e43effc4_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023040368f5dd7d44694a1cbb1af234e43effc4_del 2023-04-03 02:55:35,110 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc1ca2aa37d940b188c9bf9e2dcbb231 2023-04-03 02:55:35,111 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/.tmp/family2/fc1ca2aa37d940b188c9bf9e2dcbb231 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/fc1ca2aa37d940b188c9bf9e2dcbb231 2023-04-03 02:55:35,116 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc1ca2aa37d940b188c9bf9e2dcbb231 2023-04-03 02:55:35,117 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fb728e0e72df91a59f2a870b06ac0762/family2 of fb728e0e72df91a59f2a870b06ac0762 into fc1ca2aa37d940b188c9bf9e2dcbb231(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:35,117 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:35,121 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:35,124 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:35,127 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:35,128 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:35,135 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:55:35,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd] to archive 2023-04-03 02:55:35,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:35,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/ae44987f45f24d03af025b21477298be 2023-04-03 02:55:35,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/01e82140e7bb4fbfa46bebe602aa317a 2023-04-03 02:55:35,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/4cb00c12e6144d9aa340d51781acb91d 2023-04-03 02:55:35,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/7ec851cf529e4b98ae0b57855b36b2ab 2023-04-03 02:55:35,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family2/428c1ae45faf4f0e895b7714b5b57abd 2023-04-03 02:55:35,151 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-04-03 02:55:35,154 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023040375fed355bc3a44b3bf191912f19d76a4_del 2023-04-03 02:55:35,157 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230403e8a6e20622374e039ea919ba6d7c6c40_del 2023-04-03 02:55:35,160 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403a2d99e4c037c45028f189f48d51683d2_del 2023-04-03 02:55:35,160 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:35,161 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(139): Connect 0x50eba383 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:35,165 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1a0f9b20, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:35,167 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:35,167 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:35,167 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:35,226 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403142912cb410c44d08477e291f7568663 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230403142912cb410c44d08477e291f7568663 2023-04-03 02:55:35,226 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:35,242 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:35,245 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:35,253 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:55:35,269 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49712, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:55:35,293 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/family1/470134fbbec54b9f9615d5082bcfd04d first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-04-03 02:55:35,294 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/family1/470134fbbec54b9f9615d5082bcfd04d}] 2023-04-03 02:55:35,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/family1/470134fbbec54b9f9615d5082bcfd04d for inclusion in dc22fb15c436508e6e6e9e67d65eb6d9/family1 2023-04-03 02:55:35,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-04-03 02:55:35,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last=1 2023-04-03 02:55:35,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:55:35,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230403/family1/470134fbbec54b9f9615d5082bcfd04d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__ccepv1m61lbckor421jd4ok0ljv5ua5ibq9724puq970slk3glccd8dgorlhjid3/family1/470134fbbec54b9f9615d5082bcfd04d 2023-04-03 02:55:35,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__ccepv1m61lbckor421jd4ok0ljv5ua5ibq9724puq970slk3glccd8dgorlhjid3/family1/470134fbbec54b9f9615d5082bcfd04d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/9945e86009284e039c11bdc07ff6ab48_SeqId_28_ 2023-04-03 02:55:35,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__ccepv1m61lbckor421jd4ok0ljv5ua5ibq9724puq970slk3glccd8dgorlhjid3/family1/470134fbbec54b9f9615d5082bcfd04d into dc22fb15c436508e6e6e9e67d65eb6d9/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/9945e86009284e039c11bdc07ff6ab48_SeqId_28_ - updating store file list. 2023-04-03 02:55:35,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/9945e86009284e039c11bdc07ff6ab48_SeqId_28_ into dc22fb15c436508e6e6e9e67d65eb6d9/family1 2023-04-03 02:55:35,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__ccepv1m61lbckor421jd4ok0ljv5ua5ibq9724puq970slk3glccd8dgorlhjid3/family1/470134fbbec54b9f9615d5082bcfd04d into dc22fb15c436508e6e6e9e67d65eb6d9/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/family1/9945e86009284e039c11bdc07ff6ab48_SeqId_28_) 2023-04-03 02:55:35,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__ccepv1m61lbckor421jd4ok0ljv5ua5ibq9724puq970slk3glccd8dgorlhjid3/family1/470134fbbec54b9f9615d5082bcfd04d 2023-04-03 02:55:35,317 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:35,318 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:35,320 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304031a3baee927ae49c0942de80d7d7cf7c7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304031a3baee927ae49c0942de80d7d7cf7c7 2023-04-03 02:55:35,320 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-03 02:55:35,320 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230403' 2023-04-03 02:55:35,320 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:35,626 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304033530e198f6bc49f8a79d350416edb09a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304033530e198f6bc49f8a79d350416edb09a 2023-04-03 02:55:35,627 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:35,657 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230403/family1/c41b7375c9004d538cf0438a406fd026 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-04-03 02:55:35,657 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230403/family1/c41b7375c9004d538cf0438a406fd026}] 2023-04-03 02:55:35,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230403/family1/c41b7375c9004d538cf0438a406fd026 for inclusion in fb728e0e72df91a59f2a870b06ac0762/family1 2023-04-03 02:55:35,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-04-03 02:55:35,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-03 02:55:35,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:55:35,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230403/family1/c41b7375c9004d538cf0438a406fd026 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__649c3gfe75d35jddigroq6t6m1b55qeftomunmp4rkdulg8ineen1iffkn8j18v1/family1/c41b7375c9004d538cf0438a406fd026 2023-04-03 02:55:35,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__649c3gfe75d35jddigroq6t6m1b55qeftomunmp4rkdulg8ineen1iffkn8j18v1/family1/c41b7375c9004d538cf0438a406fd026 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/b3032e42057e4063a91a491769c22e16_SeqId_28_ 2023-04-03 02:55:35,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__649c3gfe75d35jddigroq6t6m1b55qeftomunmp4rkdulg8ineen1iffkn8j18v1/family1/c41b7375c9004d538cf0438a406fd026 into fb728e0e72df91a59f2a870b06ac0762/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/b3032e42057e4063a91a491769c22e16_SeqId_28_ - updating store file list. 2023-04-03 02:55:35,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/b3032e42057e4063a91a491769c22e16_SeqId_28_ into fb728e0e72df91a59f2a870b06ac0762/family1 2023-04-03 02:55:35,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__649c3gfe75d35jddigroq6t6m1b55qeftomunmp4rkdulg8ineen1iffkn8j18v1/family1/c41b7375c9004d538cf0438a406fd026 into fb728e0e72df91a59f2a870b06ac0762/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/family1/b3032e42057e4063a91a491769c22e16_SeqId_28_) 2023-04-03 02:55:35,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__649c3gfe75d35jddigroq6t6m1b55qeftomunmp4rkdulg8ineen1iffkn8j18v1/family1/c41b7375c9004d538cf0438a406fd026 2023-04-03 02:55:35,670 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:35,670 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:35,672 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230403268c1edf105745e59f1ec5e80da07dad to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230403268c1edf105745e59f1ec5e80da07dad 2023-04-03 02:55:35,672 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-03 02:55:35,672 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230403' 2023-04-03 02:55:35,672 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:36,026 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403b706c346414f4983b6e48c2d835dec3b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403b706c346414f4983b6e48c2d835dec3b 2023-04-03 02:55:36,027 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:36,062 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230403/family1/10a07d40d003488ebda5120e7b40b9f9 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-04-03 02:55:36,063 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230403/family1/10a07d40d003488ebda5120e7b40b9f9}] 2023-04-03 02:55:36,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230403/family1/10a07d40d003488ebda5120e7b40b9f9 for inclusion in 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 2023-04-03 02:55:36,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-04-03 02:55:36,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-03 02:55:36,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:55:36,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230403/family1/10a07d40d003488ebda5120e7b40b9f9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__7g22qc78n9eaf1mj1hgfs8fkgmergtvrtpji9u2umcs7o4hlm2vmdcfc5q0tm2q6/family1/10a07d40d003488ebda5120e7b40b9f9 2023-04-03 02:55:36,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__7g22qc78n9eaf1mj1hgfs8fkgmergtvrtpji9u2umcs7o4hlm2vmdcfc5q0tm2q6/family1/10a07d40d003488ebda5120e7b40b9f9 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/d3f6f39d607643b2819580179351f115_SeqId_28_ 2023-04-03 02:55:36,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__7g22qc78n9eaf1mj1hgfs8fkgmergtvrtpji9u2umcs7o4hlm2vmdcfc5q0tm2q6/family1/10a07d40d003488ebda5120e7b40b9f9 into 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/d3f6f39d607643b2819580179351f115_SeqId_28_ - updating store file list. 2023-04-03 02:55:36,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/d3f6f39d607643b2819580179351f115_SeqId_28_ into 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 2023-04-03 02:55:36,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__7g22qc78n9eaf1mj1hgfs8fkgmergtvrtpji9u2umcs7o4hlm2vmdcfc5q0tm2q6/family1/10a07d40d003488ebda5120e7b40b9f9 into 6e32a990bbbbb168b8d3ba4aabf4ebb2/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/family1/d3f6f39d607643b2819580179351f115_SeqId_28_) 2023-04-03 02:55:36,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__ns_testMinorCompaction__7g22qc78n9eaf1mj1hgfs8fkgmergtvrtpji9u2umcs7o4hlm2vmdcfc5q0tm2q6/family1/10a07d40d003488ebda5120e7b40b9f9 2023-04-03 02:55:36,078 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:36,079 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:36,080 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403c7aa25be200640feba5a5848e8eebda1 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230403c7aa25be200640feba5a5848e8eebda1 2023-04-03 02:55:36,080 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-03 02:55:36,080 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230403' 2023-04-03 02:55:36,081 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230403 for table='ns:testMinorCompaction' and column='family1' 2023-04-03 02:55:36,081 INFO [Listener at localhost/37595] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-04-03 02:55:36,085 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:36,087 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:36,090 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:36,091 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:36,122 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=256 (was 288), OpenFileDescriptor=637 (was 759), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=494 (was 407) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=3365 (was 3396) 2023-04-03 02:55:36,139 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=256, OpenFileDescriptor=637, MaxFileDescriptor=60000, SystemLoadAverage=494, ProcessCount=171, AvailableMemoryMB=3365 2023-04-03 02:55:36,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-03 02:55:36,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:36,144 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:55:36,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-04-03 02:55:36,144 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:55:36,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:36,146 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:36,146 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941 empty. 2023-04-03 02:55:36,147 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:36,147 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-04-03 02:55:36,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:36,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:36,556 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-03 02:55:36,557 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 04487bf6cb13446fa4432d7bb8657941, NAME => 'testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:36,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:36,963 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:36,963 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 04487bf6cb13446fa4432d7bb8657941, disabling compactions & flushes 2023-04-03 02:55:36,963 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:36,963 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:36,963 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. after waiting 0 ms 2023-04-03 02:55:36,963 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:36,964 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:36,964 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:36,966 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:55:36,967 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490536967"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490536967"}]},"ts":"1680490536967"} 2023-04-03 02:55:36,968 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-03 02:55:36,969 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:55:36,969 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490536969"}]},"ts":"1680490536969"} 2023-04-03 02:55:36,970 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-04-03 02:55:36,973 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=04487bf6cb13446fa4432d7bb8657941, ASSIGN}] 2023-04-03 02:55:36,974 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=04487bf6cb13446fa4432d7bb8657941, ASSIGN 2023-04-03 02:55:36,975 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=04487bf6cb13446fa4432d7bb8657941, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:37,126 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=04487bf6cb13446fa4432d7bb8657941, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:37,126 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490537126"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490537126"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490537126"}]},"ts":"1680490537126"} 2023-04-03 02:55:37,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 04487bf6cb13446fa4432d7bb8657941, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:37,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:37,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:37,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 04487bf6cb13446fa4432d7bb8657941, NAME => 'testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.', STARTKEY => '', ENDKEY => ''} 2023-04-03 02:55:37,282 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:37,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,284 INFO [StoreOpener-04487bf6cb13446fa4432d7bb8657941-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,285 DEBUG [StoreOpener-04487bf6cb13446fa4432d7bb8657941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:37,285 DEBUG [StoreOpener-04487bf6cb13446fa4432d7bb8657941-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:37,286 INFO [StoreOpener-04487bf6cb13446fa4432d7bb8657941-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 04487bf6cb13446fa4432d7bb8657941 columnFamilyName family1 2023-04-03 02:55:37,289 INFO [StoreOpener-04487bf6cb13446fa4432d7bb8657941-1] regionserver.HStore(310): Store=04487bf6cb13446fa4432d7bb8657941/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:37,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,290 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:55:37,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:37,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 04487bf6cb13446fa4432d7bb8657941; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67421406, jitterRate=0.004657238721847534}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-03 02:55:37,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:37,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., pid=114, masterSystemTime=1680490537279 2023-04-03 02:55:37,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:37,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:37,296 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=04487bf6cb13446fa4432d7bb8657941, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:37,297 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680490537296"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490537296"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490537296"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490537296"}]},"ts":"1680490537296"} 2023-04-03 02:55:37,299 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-04-03 02:55:37,299 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 04487bf6cb13446fa4432d7bb8657941, server=jenkins-hbase4.apache.org,45927,1680490420719 in 170 msec 2023-04-03 02:55:37,300 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-04-03 02:55:37,301 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=04487bf6cb13446fa4432d7bb8657941, ASSIGN in 326 msec 2023-04-03 02:55:37,301 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:55:37,301 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490537301"}]},"ts":"1680490537301"} 2023-04-03 02:55:37,302 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-04-03 02:55:37,304 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:55:37,305 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1630 sec 2023-04-03 02:55:38,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-03 02:55:38,250 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-04-03 02:55:38,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:38,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,274 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:38,274 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:38,274 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:38,274 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:38,275 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,275 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:38,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:38,276 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:38,277 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:38,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,284 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:38,284 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:38,284 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:38,284 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:38,286 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:38,286 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:38,286 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:38,286 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:38,287 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-03 02:55:38,722 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d5a80cc2455548b7aab65d418e01329d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d5a80cc2455548b7aab65d418e01329d 2023-04-03 02:55:39,126 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/d4434d82395c48cfba129a08f2e8f9f3 2023-04-03 02:55:39,131 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/d4434d82395c48cfba129a08f2e8f9f3 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3 2023-04-03 02:55:39,134 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3, entries=1000, sequenceid=5, filesize=166.0 K 2023-04-03 02:55:39,135 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 848ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:39,135 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:39,136 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:39,136 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:39,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:39,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:39,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:39,136 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:39,138 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,138 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,138 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,138 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:39,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:39,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41b5ec5c[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:39,140 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:39,140 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,142 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,142 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:39,142 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:39,142 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:39,142 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,142 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:39,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:39,143 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:39,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,143 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:39,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:39,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:39,146 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:39,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:39,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:39,146 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,146 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:39,147 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,147 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,147 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:39,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,148 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,149 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:39,151 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:39,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:39,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:39,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:39,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,151 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,152 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:39,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:39,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:39,152 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:39,152 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:39,252 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:39,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:39,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:39,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:39,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,258 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:39,258 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:39,258 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:39,258 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:39,259 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,259 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,260 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:39,260 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:39,260 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,260 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:39,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:39,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:39,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:39,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:39,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:39,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:39,261 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:39,261 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:39,262 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:39,262 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-03 02:55:39,677 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a2600b5be46840018d9ab745ed7468b3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2600b5be46840018d9ab745ed7468b3 2023-04-03 02:55:39,682 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/bb1a9039c63d4fc189d088105d6c5989 2023-04-03 02:55:39,686 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/bb1a9039c63d4fc189d088105d6c5989 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989 2023-04-03 02:55:39,689 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989, entries=1, sequenceid=9, filesize=5.0 K 2023-04-03 02:55:39,690 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 428ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:39,690 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:39,690 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:39,692 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,692 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,692 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,692 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@238709c0[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:39,693 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:39,693 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,695 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,696 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:39,696 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:39,696 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:39,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,696 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:39,697 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:39,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,697 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:39,699 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:39,699 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:39,699 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:39,699 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,699 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:39,700 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,700 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:39,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,700 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:39,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,702 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:39,703 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,704 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:39,704 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:39,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:39,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:39,705 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,704 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:39,705 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:39,706 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:39,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:39,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:39,806 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:39,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:39,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:39,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:39,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,813 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:39,813 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:39,813 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:39,814 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:39,814 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,814 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,818 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,818 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:39,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:39,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:39,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:39,821 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:39,821 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:39,821 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:39,821 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:39,821 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:40,236 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011cf4c6c375654168864c10207f8be67e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011cf4c6c375654168864c10207f8be67e 2023-04-03 02:55:40,641 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/2debbff3cc224475a3ede979fb972d51 2023-04-03 02:55:40,645 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/2debbff3cc224475a3ede979fb972d51 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51 2023-04-03 02:55:40,649 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51, entries=1, sequenceid=13, filesize=4.9 K 2023-04-03 02:55:40,652 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 831ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:40,652 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:40,652 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:40,652 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:40,653 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:40,653 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,653 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:40,653 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:40,654 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:40,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:40,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,654 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:40,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:40,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:40,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:40,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:40,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d2187ba[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:40,656 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:40,656 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,657 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,657 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:40,657 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:40,657 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:40,657 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,657 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:40,661 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:40,661 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:40,661 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,661 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:40,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:40,664 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:40,664 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:40,664 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:40,664 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:40,664 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,664 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:40,665 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,665 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:40,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:40,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,665 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:40,666 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:40,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:40,667 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:40,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:40,670 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:40,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:40,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:40,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:40,670 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:40,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:40,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,671 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,671 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:40,671 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:40,671 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:40,771 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:40,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:40,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:40,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:40,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,777 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:40,777 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:40,777 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:40,777 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:40,778 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,778 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,779 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:40,779 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:40,779 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:40,779 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:40,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:40,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,780 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:40,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:40,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:40,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:40,780 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:40,781 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:40,781 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:40,781 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:40,781 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:40,781 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:41,193 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057046185fa50d42bb93738e5b69ca7770 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057046185fa50d42bb93738e5b69ca7770 2023-04-03 02:55:41,198 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/f740bef5e97a4e2d8afbd0d7958d7c1b 2023-04-03 02:55:41,202 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/f740bef5e97a4e2d8afbd0d7958d7c1b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b 2023-04-03 02:55:41,205 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b, entries=1, sequenceid=17, filesize=4.9 K 2023-04-03 02:55:41,206 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 425ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:41,206 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:41,206 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:41,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:41,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:41,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:41,206 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:41,209 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,209 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,209 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,209 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:41,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fdbc139[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:41,211 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:41,211 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,211 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,212 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:41,212 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:41,212 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:41,212 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,212 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:41,213 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,213 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:41,213 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,213 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:41,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:41,217 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:41,217 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:41,217 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:41,217 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,217 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:41,218 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,218 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,219 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:41,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:41,219 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,221 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,224 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,224 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,224 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,224 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,225 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:41,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:41,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:41,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:41,225 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:41,225 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:41,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:41,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:41,325 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:41,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:41,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:41,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:41,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,332 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:41,332 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:41,332 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:41,332 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:41,332 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,332 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,333 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:41,333 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:41,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:41,334 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,334 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:41,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:41,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:41,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:41,335 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:41,335 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:41,335 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:41,335 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:41,348 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228cde880525eae4a2a9b61086af597f88b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cde880525eae4a2a9b61086af597f88b 2023-04-03 02:55:41,755 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/e7694d3b0716413e91361ecc77b998d0 2023-04-03 02:55:41,759 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/e7694d3b0716413e91361ecc77b998d0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0 2023-04-03 02:55:41,762 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0, entries=1, sequenceid=21, filesize=4.9 K 2023-04-03 02:55:41,763 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 428ms, sequenceid=21, compaction requested=true 2023-04-03 02:55:41,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:41,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:41,765 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,765 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,766 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,766 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:41,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61620b92[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:41,767 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:41,767 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,768 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,768 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:41,768 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:41,768 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:41,768 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,768 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:41,773 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:41,773 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:41,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:41,775 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:41,775 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:41,775 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:41,775 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,775 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:41,776 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,776 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:41,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:41,777 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:41,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,778 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:41,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,782 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:41,782 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:41,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:41,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:41,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:41,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:41,785 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,785 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,785 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,785 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:41,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:41,785 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,786 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,786 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,785 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:41,786 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:41,786 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:41,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:41,886 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:41,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:41,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:41,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:41,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,892 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:41,892 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:41,892 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:41,892 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:41,893 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,893 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,894 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:41,894 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:41,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,895 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,896 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:41,896 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:41,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,896 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,896 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:41,896 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:41,896 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:41,897 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:41,897 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:41,897 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:41,897 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:42,317 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312575d5b4a3d4450d85f9fd408b72d6f3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512312575d5b4a3d4450d85f9fd408b72d6f3 2023-04-03 02:55:42,334 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/1d29797bff7b4afb90d281b6f696274a 2023-04-03 02:55:42,345 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/1d29797bff7b4afb90d281b6f696274a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a 2023-04-03 02:55:42,349 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a, entries=1, sequenceid=25, filesize=4.9 K 2023-04-03 02:55:42,349 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 452ms, sequenceid=25, compaction requested=true 2023-04-03 02:55:42,349 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:42,349 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:42,349 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:42,350 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:42,350 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,350 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:42,350 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:42,351 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,351 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:42,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:42,351 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:42,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:42,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:42,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:42,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:42,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ab0d777[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:42,353 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:42,353 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,361 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,361 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:42,361 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:42,361 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:42,362 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:42,363 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:42,363 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:42,363 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:42,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,363 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:42,366 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,366 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:42,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,366 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:42,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:42,367 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,368 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:42,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:42,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:42,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:42,371 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:42,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,371 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:42,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:42,471 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:42,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:42,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:42,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:42,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,478 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:42,478 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:42,478 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:42,478 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:42,478 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,478 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,480 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:42,480 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:42,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:42,480 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,480 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:42,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:42,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,481 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:42,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,481 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:42,481 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:42,481 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:42,481 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:42,482 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:42,482 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:42,482 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:42,501 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010151088ddc030944ed88e516bc0ac3a494 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010151088ddc030944ed88e516bc0ac3a494 2023-04-03 02:55:42,844 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:42,914 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/b989e7e580cf45208b8c379b2ee55df8 2023-04-03 02:55:42,918 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/b989e7e580cf45208b8c379b2ee55df8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8 2023-04-03 02:55:42,922 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8, entries=1, sequenceid=29, filesize=4.9 K 2023-04-03 02:55:42,923 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 441ms, sequenceid=29, compaction requested=true 2023-04-03 02:55:42,923 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:42,923 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:42,923 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:42,923 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:42,923 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,923 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:42,923 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:42,924 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,924 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:42,925 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,925 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:42,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:42,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b7d83cc[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:42,926 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:42,926 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,932 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:42,933 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:42,933 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,933 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:42,934 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:42,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:42,934 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:42,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:42,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:42,936 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:42,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:42,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:42,936 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,936 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:42,938 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:42,938 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,938 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:42,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:42,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,939 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,940 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:42,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:42,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:42,942 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:42,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:42,942 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:42,943 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:42,943 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,943 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:42,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:42,943 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:42,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:43,043 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:43,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:43,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:43,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:43,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,049 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:43,049 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:43,049 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:43,049 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:43,050 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,050 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,052 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:43,052 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:43,053 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,053 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:43,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,054 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:43,054 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:43,054 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:43,054 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:43,054 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:43,054 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:43,054 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:43,055 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:43,071 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036320de15abe243568e9145a1f1002519 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036320de15abe243568e9145a1f1002519 2023-04-03 02:55:43,483 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/0b4c245313da4ccc85b26b25fce629a8 2023-04-03 02:55:43,487 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/0b4c245313da4ccc85b26b25fce629a8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8 2023-04-03 02:55:43,491 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8, entries=1, sequenceid=33, filesize=4.9 K 2023-04-03 02:55:43,492 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 437ms, sequenceid=33, compaction requested=true 2023-04-03 02:55:43,492 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:43,492 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:43,492 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:43,492 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:43,492 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,492 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:43,492 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:43,494 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,494 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:43,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:43,494 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:43,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:43,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7afe5d93[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:43,496 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:43,496 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,497 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,497 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:43,497 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:43,497 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:43,497 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,497 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:43,499 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:43,499 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:43,499 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:43,504 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:43,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:43,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:43,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:43,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:43,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:43,508 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:43,508 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:43,508 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:43,508 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,508 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:43,511 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,511 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:43,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,511 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:43,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:43,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:43,512 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,513 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:43,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,518 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:43,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:43,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:43,518 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:43,519 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:43,519 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:43,619 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:43,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:43,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:43,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:43,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,633 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:43,633 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:43,633 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:43,633 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:43,633 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,633 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,634 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,634 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:43,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:43,634 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:43,634 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:43,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:43,636 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:43,636 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:43,636 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:44,047 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f9a95cc654384b6db685dac82fcc0672 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907f9a95cc654384b6db685dac82fcc0672 2023-04-03 02:55:44,452 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/f2dac2647d0346d89f6602fcabbce2ed 2023-04-03 02:55:44,455 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/f2dac2647d0346d89f6602fcabbce2ed as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed 2023-04-03 02:55:44,458 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed, entries=1, sequenceid=37, filesize=4.9 K 2023-04-03 02:55:44,459 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 823ms, sequenceid=37, compaction requested=true 2023-04-03 02:55:44,459 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:44,459 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:44,459 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:44,459 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:44,459 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,459 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:44,459 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:44,460 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,460 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:44,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:44,460 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,461 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:44,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:44,462 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b31f7c5[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:44,462 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:44,462 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,463 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,463 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:44,463 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,463 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:44,463 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:44,463 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:44,465 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,465 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:44,465 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:44,465 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:44,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:44,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:44,467 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:44,468 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:44,468 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:44,468 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,468 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:44,469 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,469 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:44,469 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:44,469 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:44,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,470 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,472 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:44,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:44,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:44,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:44,473 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:44,472 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:44,473 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,473 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:44,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:44,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:44,573 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:44,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:44,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:44,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:44,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,579 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:44,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:44,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-03 02:55:44,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-03 02:55:44,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,579 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,580 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,580 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:44,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:44,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:44,580 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:44,581 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:44,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,581 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,581 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-03 02:55:44,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:44,582 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-03 02:55:44,582 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:44,582 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-03 02:55:44,582 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-03 02:55:44,582 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:44,582 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:44,582 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. started... 2023-04-03 02:55:44,582 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 04487bf6cb13446fa4432d7bb8657941 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-03 02:55:44,992 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b5905f4148cb4d6d85f684d4b0e9746f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5905f4148cb4d6d85f684d4b0e9746f 2023-04-03 02:55:45,396 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/de0cdf3715d74a94a54a59aab4a9624f 2023-04-03 02:55:45,400 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/de0cdf3715d74a94a54a59aab4a9624f as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f 2023-04-03 02:55:45,403 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f, entries=1, sequenceid=41, filesize=4.9 K 2023-04-03 02:55:45,404 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 04487bf6cb13446fa4432d7bb8657941 in 822ms, sequenceid=41, compaction requested=true 2023-04-03 02:55:45,404 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:45,404 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:45,404 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-03 02:55:45,404 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-03 02:55:45,404 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,404 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-03 02:55:45,404 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-03 02:55:45,407 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:45,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:45,407 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:45,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-03 02:55:45,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@774682bb[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:45,408 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-03 02:55:45,408 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,409 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,409 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-03 02:55:45,409 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-03 02:55:45,409 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:45,409 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,409 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:45,410 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,410 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-03 02:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:45,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:45,410 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:45,411 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-03 02:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:45,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-03 02:55:45,412 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-03 02:55:45,412 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:45,412 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:45,412 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,412 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:45,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:45,414 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:45,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:45,414 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:45,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,415 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:45,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:45,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-03 02:55:45,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:45,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-03 02:55:45,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:45,418 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:45,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:45,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:45,418 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:45,419 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,419 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,419 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,519 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:45,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:45,520 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-04-03 02:55:45,522 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:55:45,523 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:55:45,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:55:45,523 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491145523 2023-04-03 02:55:45,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-03 02:55:45,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-04-03 02:55:45,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:45,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x25f72ce3 to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:55:45,529 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@260e576f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:55:45,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:46,351 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030dc71c3b54bc4e96adfd3de841bf77b6 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601030dc71c3b54bc4e96adfd3de841bf77b6 2023-04-03 02:55:46,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:46,364 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:55:46,366 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36990, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:55:46,371 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:55:46,372 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50582, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:55:46,381 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/de18ac37a98745b0978e607b4d5bf69b first=Optional[r5] last=Optional[r6] 2023-04-03 02:55:46,381 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/de18ac37a98745b0978e607b4d5bf69b}] 2023-04-03 02:55:46,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/de18ac37a98745b0978e607b4d5bf69b for inclusion in 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:46,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-04-03 02:55:46,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:55:46,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:46,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/de18ac37a98745b0978e607b4d5bf69b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__h4hcp3er0b2kbih3460n4tboj28josnuscurgoorirr237qo66gu3g5gcjftbvq6/family1/de18ac37a98745b0978e607b4d5bf69b 2023-04-03 02:55:46,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__h4hcp3er0b2kbih3460n4tboj28josnuscurgoorirr237qo66gu3g5gcjftbvq6/family1/de18ac37a98745b0978e607b4d5bf69b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_ 2023-04-03 02:55:46,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__h4hcp3er0b2kbih3460n4tboj28josnuscurgoorirr237qo66gu3g5gcjftbvq6/family1/de18ac37a98745b0978e607b4d5bf69b into 04487bf6cb13446fa4432d7bb8657941/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_ - updating store file list. 2023-04-03 02:55:46,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_ into 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:46,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__h4hcp3er0b2kbih3460n4tboj28josnuscurgoorirr237qo66gu3g5gcjftbvq6/family1/de18ac37a98745b0978e607b4d5bf69b into 04487bf6cb13446fa4432d7bb8657941/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_) 2023-04-03 02:55:46,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__h4hcp3er0b2kbih3460n4tboj28josnuscurgoorirr237qo66gu3g5gcjftbvq6/family1/de18ac37a98745b0978e607b4d5bf69b 2023-04-03 02:55:46,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-03 02:55:46,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-03 02:55:46,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 04487bf6cb13446fa4432d7bb8657941/family1 is initiating minor compaction 2023-04-03 02:55:46,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@74947b07; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:46,396 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 04487bf6cb13446fa4432d7bb8657941/family1 in testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:46,397 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp, totalSize=49.7 K 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting bb1a9039c63d4fc189d088105d6c5989, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 2debbff3cc224475a3ede979fb972d51, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f740bef5e97a4e2d8afbd0d7958d7c1b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting e7694d3b0716413e91361ecc77b998d0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 1d29797bff7b4afb90d281b6f696274a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting b989e7e580cf45208b8c379b2ee55df8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 0b4c245313da4ccc85b26b25fce629a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting f2dac2647d0346d89f6602fcabbce2ed, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting de0cdf3715d74a94a54a59aab4a9624f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-03 02:55:46,397 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting dbdf4fbb7f904993ae148910316aad69_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-03 02:55:46,398 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:46,399 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:46,400 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010151088ddc030944ed88e516bc0ac3a494 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010151088ddc030944ed88e516bc0ac3a494 2023-04-03 02:55:46,401 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036320de15abe243568e9145a1f1002519 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036320de15abe243568e9145a1f1002519 2023-04-03 02:55:46,401 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-04-03 02:55:46,401 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-04-03 02:55:46,402 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:46,402 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:46,420 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 04487bf6cb13446fa4432d7bb8657941#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-04-03 02:55:47,239 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/2aa99db2979542348a3ce157f2b48a82 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2aa99db2979542348a3ce157f2b48a82 2023-04-03 02:55:47,243 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310d53b7be38e74f7d8a57c3f03e039cfe to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512310d53b7be38e74f7d8a57c3f03e039cfe 2023-04-03 02:55:47,243 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 04487bf6cb13446fa4432d7bb8657941/family1 of 04487bf6cb13446fa4432d7bb8657941 into 2aa99db2979542348a3ce157f2b48a82(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:47,243 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:47,243 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1680490546396; duration=0sec 2023-04-03 02:55:47,243 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:47,243 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:47,246 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_] to archive 2023-04-03 02:55:47,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:47,248 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/bb1a9039c63d4fc189d088105d6c5989 2023-04-03 02:55:47,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2debbff3cc224475a3ede979fb972d51 2023-04-03 02:55:47,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f740bef5e97a4e2d8afbd0d7958d7c1b 2023-04-03 02:55:47,251 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/e7694d3b0716413e91361ecc77b998d0 2023-04-03 02:55:47,253 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1d29797bff7b4afb90d281b6f696274a 2023-04-03 02:55:47,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/b989e7e580cf45208b8c379b2ee55df8 2023-04-03 02:55:47,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/0b4c245313da4ccc85b26b25fce629a8 2023-04-03 02:55:47,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/f2dac2647d0346d89f6602fcabbce2ed 2023-04-03 02:55:47,257 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/de0cdf3715d74a94a54a59aab4a9624f 2023-04-03 02:55:47,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/dbdf4fbb7f904993ae148910316aad69_SeqId_44_ 2023-04-03 02:55:47,267 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/487ee1aabcf94c039a10e70873c006f4 first=Optional[r1] last=Optional[r4] 2023-04-03 02:55:47,268 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/487ee1aabcf94c039a10e70873c006f4}] 2023-04-03 02:55:47,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/487ee1aabcf94c039a10e70873c006f4 for inclusion in 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:47,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-04-03 02:55:47,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:55:47,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:47,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/487ee1aabcf94c039a10e70873c006f4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdansv15qvp7nmu3lf40h55h5h3dk3e2sos6shbb0pqgbmlp607k43it2lbpmt61/family1/487ee1aabcf94c039a10e70873c006f4 2023-04-03 02:55:47,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdansv15qvp7nmu3lf40h55h5h3dk3e2sos6shbb0pqgbmlp607k43it2lbpmt61/family1/487ee1aabcf94c039a10e70873c006f4 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_ 2023-04-03 02:55:47,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdansv15qvp7nmu3lf40h55h5h3dk3e2sos6shbb0pqgbmlp607k43it2lbpmt61/family1/487ee1aabcf94c039a10e70873c006f4 into 04487bf6cb13446fa4432d7bb8657941/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_ - updating store file list. 2023-04-03 02:55:47,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_ into 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:47,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdansv15qvp7nmu3lf40h55h5h3dk3e2sos6shbb0pqgbmlp607k43it2lbpmt61/family1/487ee1aabcf94c039a10e70873c006f4 into 04487bf6cb13446fa4432d7bb8657941/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_) 2023-04-03 02:55:47,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdansv15qvp7nmu3lf40h55h5h3dk3e2sos6shbb0pqgbmlp607k43it2lbpmt61/family1/487ee1aabcf94c039a10e70873c006f4 2023-04-03 02:55:47,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-03 02:55:47,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-03 02:55:47,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(1912): 04487bf6cb13446fa4432d7bb8657941/family1 is initiating minor compaction (all files) 2023-04-03 02:55:47,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4d28b0c7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:47,279 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 04487bf6cb13446fa4432d7bb8657941/family1 in testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:55:47,279 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2aa99db2979542348a3ce157f2b48a82, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp, totalSize=178.1 K 2023-04-03 02:55:47,280 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting d4434d82395c48cfba129a08f2e8f9f3, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878844911 2023-04-03 02:55:47,280 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 2aa99db2979542348a3ce157f2b48a82, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621244911 2023-04-03 02:55:47,280 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] compactions.Compactor(207): Compacting 2c994e400a3f4fff835c409c10b268fb_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965244911 2023-04-03 02:55:47,281 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:47,281 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:47,283 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011cf4c6c375654168864c10207f8be67e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011cf4c6c375654168864c10207f8be67e 2023-04-03 02:55:47,284 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057046185fa50d42bb93738e5b69ca7770 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512057046185fa50d42bb93738e5b69ca7770 2023-04-03 02:55:47,285 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cde880525eae4a2a9b61086af597f88b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228cde880525eae4a2a9b61086af597f88b 2023-04-03 02:55:47,286 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512312575d5b4a3d4450d85f9fd408b72d6f3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512312575d5b4a3d4450d85f9fd408b72d6f3 2023-04-03 02:55:47,286 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-04-03 02:55:47,286 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-04-03 02:55:47,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:47,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:47,294 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] throttle.PressureAwareThroughputController(145): 04487bf6cb13446fa4432d7bb8657941#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-04-03 02:55:47,313 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:55:47,313 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50eba383 to 127.0.0.1:52880 2023-04-03 02:55:47,313 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:47,313 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:55:47,314 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28b43b23 to 127.0.0.1:52880 2023-04-03 02:55:47,314 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:55:47,338 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309a9db6e6eb0341b7a81f8ea8b8c45b1a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511309a9db6e6eb0341b7a81f8ea8b8c45b1a 2023-04-03 02:55:47,339 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:47,361 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc488fa2e90d4c30b7d295dff51acee0 first=Optional[r00] last=Optional[r8] 2023-04-03 02:55:47,362 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc488fa2e90d4c30b7d295dff51acee0}] 2023-04-03 02:55:47,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc488fa2e90d4c30b7d295dff51acee0 for inclusion in 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:47,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-04-03 02:55:47,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last= 2023-04-03 02:55:47,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:47,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/cc488fa2e90d4c30b7d295dff51acee0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c2gasmjk93ngq67d34ggksebj0c4ruj3638bhej8jekk5bqai90vc9uhidheat5g/family1/cc488fa2e90d4c30b7d295dff51acee0 2023-04-03 02:55:47,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c2gasmjk93ngq67d34ggksebj0c4ruj3638bhej8jekk5bqai90vc9uhidheat5g/family1/cc488fa2e90d4c30b7d295dff51acee0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/255c1c0b71814946ba078dec82231130_SeqId_49_ 2023-04-03 02:55:47,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c2gasmjk93ngq67d34ggksebj0c4ruj3638bhej8jekk5bqai90vc9uhidheat5g/family1/cc488fa2e90d4c30b7d295dff51acee0 into 04487bf6cb13446fa4432d7bb8657941/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/255c1c0b71814946ba078dec82231130_SeqId_49_ - updating store file list. 2023-04-03 02:55:47,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/255c1c0b71814946ba078dec82231130_SeqId_49_ into 04487bf6cb13446fa4432d7bb8657941/family1 2023-04-03 02:55:47,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c2gasmjk93ngq67d34ggksebj0c4ruj3638bhej8jekk5bqai90vc9uhidheat5g/family1/cc488fa2e90d4c30b7d295dff51acee0 into 04487bf6cb13446fa4432d7bb8657941/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/255c1c0b71814946ba078dec82231130_SeqId_49_) 2023-04-03 02:55:47,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMinorCompactionWithMonthlyPolicy__c2gasmjk93ngq67d34ggksebj0c4ruj3638bhej8jekk5bqai90vc9uhidheat5g/family1/cc488fa2e90d4c30b7d295dff51acee0 2023-04-03 02:55:47,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:47,373 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:47,374 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5905f4148cb4d6d85f684d4b0e9746f to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5905f4148cb4d6d85f684d4b0e9746f 2023-04-03 02:55:47,375 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2600b5be46840018d9ab745ed7468b3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2600b5be46840018d9ab745ed7468b3 2023-04-03 02:55:47,376 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d5a80cc2455548b7aab65d418e01329d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130d5a80cc2455548b7aab65d418e01329d 2023-04-03 02:55:47,376 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-04-03 02:55:47,376 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-03 02:55:47,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:47,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:47,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-03 02:55:47,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-03 02:55:47,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:55:47,378 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-03 02:55:47,380 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8580 sec 2023-04-03 02:55:47,559 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=262 (was 256) - Thread LEAK? -, OpenFileDescriptor=647 (was 637) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=470 (was 494), ProcessCount=170 (was 171), AvailableMemoryMB=3332 (was 3365) 2023-04-03 02:55:47,570 INFO [Listener at localhost/37595] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=262, OpenFileDescriptor=647, MaxFileDescriptor=60000, SystemLoadAverage=470, ProcessCount=170, AvailableMemoryMB=3331 2023-04-03 02:55:47,590 DEBUG [Listener at localhost/37595] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-04-03 02:55:47,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10g\x99\x8D\x8Eef\xE00\xC3\xA5\x03\x1B\x9B\xFE\xE6\xF7"\x10\x98\xBAap5\x99\xD3\x0E\x9E\xDB\x00\x1F\x5C\xE8Q\xEC*\x10J\xC2\xAC\xA8\xD3Z\x0A\xD2\x05AY\x0Du\xBF\xD6'2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-03 02:55:47,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-04-03 02:55:47,644 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-04-03 02:55:47,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-04-03 02:55:47,645 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-03 02:55:47,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b empty. 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7 empty. 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be empty. 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:47,647 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:47,647 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-04-03 02:55:47,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:47,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:48,054 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-04-03 02:55:48,055 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 66d14b50c01d6d8987769782f72db5e7, NAME => 'testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10g\x99\x8D\x8Eef\xE00\xC3\xA5\x03\x1B\x9B\xFE\xE6\xF7"\x10\x98\xBAap5\x99\xD3\x0E\x9E\xDB\x00\x1F\x5C\xE8Q\xEC*\x10J\xC2\xAC\xA8\xD3Z\x0A\xD2\x05AY\x0Du\xBF\xD6'2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:48,055 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 672a7d65ccbb850591a80e29cf7d2b1b, NAME => 'testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10g\x99\x8D\x8Eef\xE00\xC3\xA5\x03\x1B\x9B\xFE\xE6\xF7"\x10\x98\xBAap5\x99\xD3\x0E\x9E\xDB\x00\x1F\x5C\xE8Q\xEC*\x10J\xC2\xAC\xA8\xD3Z\x0A\xD2\x05AY\x0Du\xBF\xD6'2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:48,055 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => a8fff145c4646476fb737ed5f93141be, NAME => 'testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10g\x99\x8D\x8Eef\xE00\xC3\xA5\x03\x1B\x9B\xFE\xE6\xF7"\x10\x98\xBAap5\x99\xD3\x0E\x9E\xDB\x00\x1F\x5C\xE8Q\xEC*\x10J\xC2\xAC\xA8\xD3Z\x0A\xD2\x05AY\x0Du\xBF\xD6'2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/.tmp 2023-04-03 02:55:48,126 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/.tmp/family1/1889b946ba694f76bd785fa5ac6095dd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/1889b946ba694f76bd785fa5ac6095dd 2023-04-03 02:55:48,130 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 04487bf6cb13446fa4432d7bb8657941/family1 of 04487bf6cb13446fa4432d7bb8657941 into 1889b946ba694f76bd785fa5ac6095dd(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:48,131 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:55:48,131 INFO [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1680490547279; duration=0sec 2023-04-03 02:55:48,131 DEBUG [RS:0;jenkins-hbase4:45927-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-03 02:55:48,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2aa99db2979542348a3ce157f2b48a82, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_] to archive 2023-04-03 02:55:48,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:48,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/d4434d82395c48cfba129a08f2e8f9f3 2023-04-03 02:55:48,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2aa99db2979542348a3ce157f2b48a82 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2aa99db2979542348a3ce157f2b48a82 2023-04-03 02:55:48,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_ to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/family1/2c994e400a3f4fff835c409c10b268fb_SeqId_47_ 2023-04-03 02:55:48,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 672a7d65ccbb850591a80e29cf7d2b1b, disabling compactions & flushes 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing a8fff145c4646476fb737ed5f93141be, disabling compactions & flushes 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 66d14b50c01d6d8987769782f72db5e7, disabling compactions & flushes 2023-04-03 02:55:48,466 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,466 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,466 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. after waiting 0 ms 2023-04-03 02:55:48,466 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,467 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. after waiting 0 ms 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. after waiting 0 ms 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,467 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:48,467 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,467 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:48,469 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-04-03 02:55:48,470 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680490548469"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490548469"}]},"ts":"1680490548469"} 2023-04-03 02:55:48,470 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548469"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490548469"}]},"ts":"1680490548469"} 2023-04-03 02:55:48,470 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548469"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680490548469"}]},"ts":"1680490548469"} 2023-04-03 02:55:48,471 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-03 02:55:48,472 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-03 02:55:48,472 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490548472"}]},"ts":"1680490548472"} 2023-04-03 02:55:48,473 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-04-03 02:55:48,476 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=672a7d65ccbb850591a80e29cf7d2b1b, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a8fff145c4646476fb737ed5f93141be, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=66d14b50c01d6d8987769782f72db5e7, ASSIGN}] 2023-04-03 02:55:48,478 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=672a7d65ccbb850591a80e29cf7d2b1b, ASSIGN 2023-04-03 02:55:48,478 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=66d14b50c01d6d8987769782f72db5e7, ASSIGN 2023-04-03 02:55:48,478 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a8fff145c4646476fb737ed5f93141be, ASSIGN 2023-04-03 02:55:48,478 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=672a7d65ccbb850591a80e29cf7d2b1b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:48,478 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=66d14b50c01d6d8987769782f72db5e7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:48,478 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a8fff145c4646476fb737ed5f93141be, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45927,1680490420719; forceNewPlan=false, retain=false 2023-04-03 02:55:48,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=66d14b50c01d6d8987769782f72db5e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,630 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=672a7d65ccbb850591a80e29cf7d2b1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,630 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a8fff145c4646476fb737ed5f93141be, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,630 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490548630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490548630"}]},"ts":"1680490548630"} 2023-04-03 02:55:48,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490548630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490548630"}]},"ts":"1680490548630"} 2023-04-03 02:55:48,630 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680490548630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1680490548630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680490548630"}]},"ts":"1680490548630"} 2023-04-03 02:55:48,632 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 672a7d65ccbb850591a80e29cf7d2b1b, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:48,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 66d14b50c01d6d8987769782f72db5e7, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:48,633 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure a8fff145c4646476fb737ed5f93141be, server=jenkins-hbase4.apache.org,45927,1680490420719}] 2023-04-03 02:55:48,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:48,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a8fff145c4646476fb737ed5f93141be, NAME => 'testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.', STARTKEY => '1', ENDKEY => '2'} 2023-04-03 02:55:48,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,807 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,808 DEBUG [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1 2023-04-03 02:55:48,808 DEBUG [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1 2023-04-03 02:55:48,808 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8fff145c4646476fb737ed5f93141be columnFamilyName family1 2023-04-03 02:55:48,810 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] regionserver.HStore(310): Store=a8fff145c4646476fb737ed5f93141be/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,810 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,811 DEBUG [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2 2023-04-03 02:55:48,811 DEBUG [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2 2023-04-03 02:55:48,811 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a8fff145c4646476fb737ed5f93141be columnFamilyName family2 2023-04-03 02:55:48,812 INFO [StoreOpener-a8fff145c4646476fb737ed5f93141be-1] regionserver.HStore(310): Store=a8fff145c4646476fb737ed5f93141be/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:48,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a8fff145c4646476fb737ed5f93141be 2023-04-03 02:55:48,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:48,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a8fff145c4646476fb737ed5f93141be; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62838184, jitterRate=-0.06363809108734131}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:48,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:48,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be., pid=122, masterSystemTime=1680490548783 2023-04-03 02:55:48,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:48,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 66d14b50c01d6d8987769782f72db5e7, NAME => 'testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.', STARTKEY => '2', ENDKEY => ''} 2023-04-03 02:55:48,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,819 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a8fff145c4646476fb737ed5f93141be, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680490548819"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490548819"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490548819"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490548819"}]},"ts":"1680490548819"} 2023-04-03 02:55:48,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-04-03 02:55:48,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure a8fff145c4646476fb737ed5f93141be, server=jenkins-hbase4.apache.org,45927,1680490420719 in 187 msec 2023-04-03 02:55:48,822 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a8fff145c4646476fb737ed5f93141be, ASSIGN in 346 msec 2023-04-03 02:55:48,839 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,840 DEBUG [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1 2023-04-03 02:55:48,841 DEBUG [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1 2023-04-03 02:55:48,841 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66d14b50c01d6d8987769782f72db5e7 columnFamilyName family1 2023-04-03 02:55:48,841 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] regionserver.HStore(310): Store=66d14b50c01d6d8987769782f72db5e7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,842 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,842 DEBUG [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2 2023-04-03 02:55:48,843 DEBUG [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2 2023-04-03 02:55:48,843 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66d14b50c01d6d8987769782f72db5e7 columnFamilyName family2 2023-04-03 02:55:48,843 INFO [StoreOpener-66d14b50c01d6d8987769782f72db5e7-1] regionserver.HStore(310): Store=66d14b50c01d6d8987769782f72db5e7/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:48,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:55:48,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:48,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 66d14b50c01d6d8987769782f72db5e7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67214069, jitterRate=0.0015676766633987427}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:48,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:48,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7., pid=121, masterSystemTime=1680490548783 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:48,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 672a7d65ccbb850591a80e29cf7d2b1b, NAME => 'testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.', STARTKEY => '', ENDKEY => '1'} 2023-04-03 02:55:48,850 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=66d14b50c01d6d8987769782f72db5e7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-03 02:55:48,850 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548850"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490548850"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490548850"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490548850"}]},"ts":"1680490548850"} 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,853 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-04-03 02:55:48,853 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 66d14b50c01d6d8987769782f72db5e7, server=jenkins-hbase4.apache.org,45927,1680490420719 in 219 msec 2023-04-03 02:55:48,853 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=66d14b50c01d6d8987769782f72db5e7, ASSIGN in 377 msec 2023-04-03 02:55:48,871 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,872 DEBUG [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1 2023-04-03 02:55:48,872 DEBUG [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1 2023-04-03 02:55:48,872 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 672a7d65ccbb850591a80e29cf7d2b1b columnFamilyName family1 2023-04-03 02:55:48,873 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] regionserver.HStore(310): Store=672a7d65ccbb850591a80e29cf7d2b1b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,873 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,874 DEBUG [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2 2023-04-03 02:55:48,874 DEBUG [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2 2023-04-03 02:55:48,874 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 672a7d65ccbb850591a80e29cf7d2b1b columnFamilyName family2 2023-04-03 02:55:48,875 INFO [StoreOpener-672a7d65ccbb850591a80e29cf7d2b1b-1] regionserver.HStore(310): Store=672a7d65ccbb850591a80e29cf7d2b1b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-03 02:55:48,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-03 02:55:48,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:55:48,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-03 02:55:48,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 672a7d65ccbb850591a80e29cf7d2b1b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73854484, jitterRate=0.10051757097244263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-03 02:55:48,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:48,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b., pid=120, masterSystemTime=1680490548783 2023-04-03 02:55:48,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:48,881 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=672a7d65ccbb850591a80e29cf7d2b1b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:48,881 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1680490548881"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1680490548881"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680490548881"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680490548881"}]},"ts":"1680490548881"} 2023-04-03 02:55:48,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-04-03 02:55:48,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 672a7d65ccbb850591a80e29cf7d2b1b, server=jenkins-hbase4.apache.org,45927,1680490420719 in 250 msec 2023-04-03 02:55:48,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-04-03 02:55:48,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=672a7d65ccbb850591a80e29cf7d2b1b, ASSIGN in 407 msec 2023-04-03 02:55:48,885 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-03 02:55:48,885 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680490548885"}]},"ts":"1680490548885"} 2023-04-03 02:55:48,886 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-04-03 02:55:48,887 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-04-03 02:55:48,888 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2460 sec 2023-04-03 02:55:48,902 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-04-03 02:55:49,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-03 02:55:49,750 INFO [Listener at localhost/37595] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-04-03 02:55:49,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:49,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:49,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:49,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:49,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-03 02:55:49,759 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-03 02:55:49,759 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:49,759 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-03 02:55:49,760 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-03 02:55:49,760 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:49,760 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:49,761 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:49,761 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:49,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:49,761 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:49,761 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:49,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:49,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:49,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:49,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-03 02:55:49,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-03 02:55:49,762 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-03 02:55:49,762 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:49,762 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-03 02:55:49,762 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. started... 2023-04-03 02:55:49,763 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:49,763 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:49,763 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66d14b50c01d6d8987769782f72db5e7 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. started... 2023-04-03 02:55:49,763 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. started... 2023-04-03 02:55:49,765 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a8fff145c4646476fb737ed5f93141be 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:49,765 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 672a7d65ccbb850591a80e29cf7d2b1b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-03 02:55:50,207 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403604ed7e3e21e4f80b7530c38447e1263 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403604ed7e3e21e4f80b7530c38447e1263 2023-04-03 02:55:50,212 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304037584cd86ced944178ace9c95cace18c0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037584cd86ced944178ace9c95cace18c0 2023-04-03 02:55:50,214 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304034c8d75236c424ef3b0976c76d0ee095d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304034c8d75236c424ef3b0976c76d0ee095d 2023-04-03 02:55:50,221 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/8fedeebe6f7e4de5a8b229b1a281fe43 2023-04-03 02:55:50,231 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0 2023-04-03 02:55:50,254 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403a9e5cdf00f384cc288b55ffa9f96ccce to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230403a9e5cdf00f384cc288b55ffa9f96ccce 2023-04-03 02:55:50,626 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/ef197984ab0f48cba90f9b0189b1b644 2023-04-03 02:55:50,658 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/aca4566e080f4a2ebd90a9c108c2b016 2023-04-03 02:55:50,662 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/8fedeebe6f7e4de5a8b229b1a281fe43 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43 2023-04-03 02:55:50,662 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040378e73dc6b32a4af3aaa4c598e333c8d0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023040378e73dc6b32a4af3aaa4c598e333c8d0 2023-04-03 02:55:50,685 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43, entries=4, sequenceid=5, filesize=5.5 K 2023-04-03 02:55:50,686 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/aca4566e080f4a2ebd90a9c108c2b016 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016 2023-04-03 02:55:50,689 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:50,690 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a8fff145c4646476fb737ed5f93141be in 926ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:50,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:50,691 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:50,691 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:51,060 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403ff153a86d34f4a04b5d70140b0b91d05 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230403ff153a86d34f4a04b5d70140b0b91d05 2023-04-03 02:55:51,067 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/edd83306bf0440fca60d891d36bf8a9d 2023-04-03 02:55:51,070 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0 2023-04-03 02:55:51,093 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0, entries=4, sequenceid=5, filesize=5.5 K 2023-04-03 02:55:51,093 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/edd83306bf0440fca60d891d36bf8a9d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d 2023-04-03 02:55:51,096 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:51,097 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 66d14b50c01d6d8987769782f72db5e7 in 1334ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:51,097 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:51,097 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:51,097 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:51,464 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/4df7be20f0a9436384d89c26f3508cf0 2023-04-03 02:55:51,468 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/ef197984ab0f48cba90f9b0189b1b644 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644 2023-04-03 02:55:51,491 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644, entries=4, sequenceid=5, filesize=5.5 K 2023-04-03 02:55:51,492 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/4df7be20f0a9436384d89c26f3508cf0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0 2023-04-03 02:55:51,494 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0, entries=2, sequenceid=5, filesize=5.1 K 2023-04-03 02:55:51,495 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 672a7d65ccbb850591a80e29cf7d2b1b in 1730ms, sequenceid=5, compaction requested=false 2023-04-03 02:55:51,495 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:51,495 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-03 02:55:51,495 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-03 02:55:51,501 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,501 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:51,501 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,501 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:51,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:51,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-03 02:55:51,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30e21c7c[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:51,502 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-03 02:55:51,502 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,503 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,503 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-03 02:55:51,503 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-03 02:55:51,503 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:51,503 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,503 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:51,505 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-03 02:55:51,505 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:51,505 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,505 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:51,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:51,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-03 02:55:51,507 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-03 02:55:51,507 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:51,507 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:51,507 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-03 02:55:51,507 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:51,509 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,509 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:51,509 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:51,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:51,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:51,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:51,510 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:51,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:51,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,511 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:51,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:51,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:51,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-03 02:55:51,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:51,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:51,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:51,512 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:51,512 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,513 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,513 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:51,513 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,513 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:51,513 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:51,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:51,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:51,613 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:51,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:51,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:51,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:51,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-03 02:55:51,619 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-03 02:55:51,619 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:51,619 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-03 02:55:51,619 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-03 02:55:51,619 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,619 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:51,620 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,620 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:51,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:51,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:51,620 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:51,620 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-03 02:55:51,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-03 02:55:51,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-03 02:55:51,621 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-03 02:55:51,621 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:51,621 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. started... 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. started... 2023-04-03 02:55:51,622 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66d14b50c01d6d8987769782f72db5e7 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:51,622 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a8fff145c4646476fb737ed5f93141be 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:51,622 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:51,622 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:51,622 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. started... 2023-04-03 02:55:51,622 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 672a7d65ccbb850591a80e29cf7d2b1b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-03 02:55:51,684 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403df51cacde9494d159ce5454b1d739012 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403df51cacde9494d159ce5454b1d739012 2023-04-03 02:55:52,065 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304035f6d8a35cbc94260b041f46ea9095f19 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304035f6d8a35cbc94260b041f46ea9095f19 2023-04-03 02:55:52,066 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304038f9c1546551b438a940c23f770f25a39 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304038f9c1546551b438a940c23f770f25a39 2023-04-03 02:55:52,103 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/d4aa62dd96a04a1bbc5ed42f61824fac 2023-04-03 02:55:52,480 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/23c047fd96e04d268ba9922bae60cfcb 2023-04-03 02:55:52,480 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/47ab455516bc4be6ab33b0236680f07a 2023-04-03 02:55:52,533 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403ba748418625340a89030449422c87320 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230403ba748418625340a89030449422c87320 2023-04-03 02:55:52,913 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304036bda39f4cc524871a26839d41201a892 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202304036bda39f4cc524871a26839d41201a892 2023-04-03 02:55:52,913 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403b1b8ed246ae04aa8b0c53bed42a47154 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230403b1b8ed246ae04aa8b0c53bed42a47154 2023-04-03 02:55:52,937 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/2c285546e0874788b33aa8fd6cb3e943 2023-04-03 02:55:52,940 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/d4aa62dd96a04a1bbc5ed42f61824fac as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac 2023-04-03 02:55:52,965 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac, entries=4, sequenceid=9, filesize=5.5 K 2023-04-03 02:55:52,966 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/2c285546e0874788b33aa8fd6cb3e943 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943 2023-04-03 02:55:52,969 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:52,969 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 672a7d65ccbb850591a80e29cf7d2b1b in 1347ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:52,970 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:52,970 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:52,970 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:53,319 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/c45e5c106b8144b1a55fa53e756ce6a9 2023-04-03 02:55:53,319 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/2e90b588ea2f4102b9b7da19e43592ca 2023-04-03 02:55:53,322 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/47ab455516bc4be6ab33b0236680f07a as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a 2023-04-03 02:55:53,322 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/23c047fd96e04d268ba9922bae60cfcb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb 2023-04-03 02:55:53,344 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a, entries=4, sequenceid=9, filesize=5.5 K 2023-04-03 02:55:53,345 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb, entries=4, sequenceid=9, filesize=5.5 K 2023-04-03 02:55:53,345 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/2e90b588ea2f4102b9b7da19e43592ca as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca 2023-04-03 02:55:53,345 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/c45e5c106b8144b1a55fa53e756ce6a9 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9 2023-04-03 02:55:53,349 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:53,350 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a8fff145c4646476fb737ed5f93141be in 1728ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:53,350 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:53,350 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:53,350 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:53,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9, entries=2, sequenceid=9, filesize=5.1 K 2023-04-03 02:55:53,351 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 66d14b50c01d6d8987769782f72db5e7 in 1729ms, sequenceid=9, compaction requested=false 2023-04-03 02:55:53,351 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:53,352 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:53,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:53,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:53,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-03 02:55:53,352 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-03 02:55:53,354 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,354 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:53,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:53,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-03 02:55:53,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3878cd36[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:53,356 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-03 02:55:53,356 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,357 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,357 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-03 02:55:53,357 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-03 02:55:53,357 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:53,357 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,357 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:53,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-03 02:55:53,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:53,358 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,358 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-03 02:55:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:53,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:53,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-03 02:55:53,360 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-03 02:55:53,360 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:53,360 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:53,360 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-03 02:55:53,360 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:53,362 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,362 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:53,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,362 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:53,362 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,363 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:53,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:53,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-03 02:55:53,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:53,367 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:53,367 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:53,467 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:53,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:53,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:53,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:53,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:53,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:53,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:53,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-03 02:55:53,473 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-03 02:55:53,473 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:53,473 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-03 02:55:53,473 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-03 02:55:53,473 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,473 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:53,474 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:53,474 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:53,475 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:53,475 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:53,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-03 02:55:53,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-03 02:55:53,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-03 02:55:53,476 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-03 02:55:53,476 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:53,476 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. started... 2023-04-03 02:55:53,476 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:53,476 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66d14b50c01d6d8987769782f72db5e7 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. started... 2023-04-03 02:55:53,476 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:53,476 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. started... 2023-04-03 02:55:53,476 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 672a7d65ccbb850591a80e29cf7d2b1b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:53,476 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a8fff145c4646476fb737ed5f93141be 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-03 02:55:53,517 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304035108c26f53204e80bd1a5c6285844328 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304035108c26f53204e80bd1a5c6285844328 2023-04-03 02:55:53,522 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403aac74ab4ec3646398577cc4f91725593 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403aac74ab4ec3646398577cc4f91725593 2023-04-03 02:55:53,925 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304030b9a691d629548c4819b447d976a4fae to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304030b9a691d629548c4819b447d976a4fae 2023-04-03 02:55:53,932 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/bba514dd5d9744caafaafd6314594a01 2023-04-03 02:55:53,936 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/7bcf502f1bc24a9092c2158fbe37c62d 2023-04-03 02:55:54,337 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-03 02:55:54,340 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/20fdd6df0a984880a75aabec561872ca 2023-04-03 02:55:54,367 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304037b71fc3c3a7f49b29f46c1781ef2b9c4 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202304037b71fc3c3a7f49b29f46c1781ef2b9c4 2023-04-03 02:55:54,378 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403f30d41d5bdd04c4ca6da86f6bdc28f51 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230403f30d41d5bdd04c4ca6da86f6bdc28f51 2023-04-03 02:55:54,771 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/6e16033fdc2e48c3996235747be10232 2023-04-03 02:55:54,772 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304034cf96a8589ed48b880a3f013252d990b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202304034cf96a8589ed48b880a3f013252d990b 2023-04-03 02:55:54,775 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/bba514dd5d9744caafaafd6314594a01 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01 2023-04-03 02:55:54,777 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/0edceddd2fc04932b9d9d41f9c4ee3d2 2023-04-03 02:55:54,781 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/20fdd6df0a984880a75aabec561872ca as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca 2023-04-03 02:55:54,782 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/83d2a0e0f00048fbaa78c2d8885a773b 2023-04-03 02:55:54,786 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/7bcf502f1bc24a9092c2158fbe37c62d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d 2023-04-03 02:55:54,801 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01, entries=4, sequenceid=13, filesize=5.5 K 2023-04-03 02:55:54,802 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/6e16033fdc2e48c3996235747be10232 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232 2023-04-03 02:55:54,805 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca, entries=4, sequenceid=13, filesize=5.5 K 2023-04-03 02:55:54,806 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/0edceddd2fc04932b9d9d41f9c4ee3d2 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2 2023-04-03 02:55:54,807 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:54,808 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 672a7d65ccbb850591a80e29cf7d2b1b in 1332ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:54,808 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:54,808 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:54,808 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:54,811 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d, entries=4, sequenceid=13, filesize=5.5 K 2023-04-03 02:55:54,811 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:54,812 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/83d2a0e0f00048fbaa78c2d8885a773b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b 2023-04-03 02:55:54,812 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a8fff145c4646476fb737ed5f93141be in 1336ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:54,812 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:54,812 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:54,812 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:54,815 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b, entries=2, sequenceid=13, filesize=5.1 K 2023-04-03 02:55:54,815 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 66d14b50c01d6d8987769782f72db5e7 in 1339ms, sequenceid=13, compaction requested=true 2023-04-03 02:55:54,816 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:54,816 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:54,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:54,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:54,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-03 02:55:54,816 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-03 02:55:54,817 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,817 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:54,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:54,818 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:54,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:54,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:54,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-03 02:55:54,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cf38a23[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:54,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-03 02:55:54,819 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,820 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-03 02:55:54,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-03 02:55:54,820 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:54,820 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,820 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:54,822 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-03 02:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-03 02:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:54,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-03 02:55:54,825 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-03 02:55:54,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:54,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:54,825 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-03 02:55:54,825 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:54,826 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,826 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:54,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:54,826 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:54,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:54,826 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,828 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:54,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,829 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-03 02:55:54,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:54,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:54,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:54,830 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:54,830 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:54,930 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:54,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:54,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:54,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:54,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. with WAL disabled. Data may be lost in the event of a crash. 2023-04-03 02:55:54,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:54,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:54,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-03 02:55:54,937 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-03 02:55:54,937 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:54,937 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-03 02:55:54,937 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-03 02:55:54,937 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,937 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:54,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,943 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:54,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:54,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:54,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:54,943 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-03 02:55:54,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-03 02:55:54,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-03 02:55:54,944 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-03 02:55:54,944 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:54,944 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:54,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:54,945 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. started... 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. started... 2023-04-03 02:55:54,945 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. started... 2023-04-03 02:55:54,945 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 672a7d65ccbb850591a80e29cf7d2b1b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:54,945 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a8fff145c4646476fb737ed5f93141be 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:54,945 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66d14b50c01d6d8987769782f72db5e7 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-03 02:55:55,012 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304037c650f0304bb419a96fda88f00b7af38 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037c650f0304bb419a96fda88f00b7af38 2023-04-03 02:55:55,015 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403852e69a5fff643d3abab0296b2e51978 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403852e69a5fff643d3abab0296b2e51978 2023-04-03 02:55:55,044 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb 2023-04-03 02:55:55,412 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230403a7cb7a69e94848cd858432b499891a00 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403a7cb7a69e94848cd858432b499891a00 2023-04-03 02:55:55,436 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/3e2f4cd4530341c5b079e1752dcc0178 2023-04-03 02:55:55,481 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403178d302e0cfe44429d1af677456112db to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230403178d302e0cfe44429d1af677456112db 2023-04-03 02:55:55,491 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403b43a4e9fd540480ab50f7f9b5da28027 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230403b43a4e9fd540480ab50f7f9b5da28027 2023-04-03 02:55:55,496 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/3af498a5620e4fa093a902109d649a04 2023-04-03 02:55:55,502 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb 2023-04-03 02:55:55,526 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb, entries=4, sequenceid=17, filesize=5.5 K 2023-04-03 02:55:55,527 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/3af498a5620e4fa093a902109d649a04 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04 2023-04-03 02:55:55,530 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:55,531 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 66d14b50c01d6d8987769782f72db5e7 in 586ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:55,531 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:55,531 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:55,531 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:55,862 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/a6b2d4aa1d4645d28805abcb793c08f8 2023-04-03 02:55:55,887 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/75a78379469748d9b19b9584bf665bc7 2023-04-03 02:55:55,893 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/3e2f4cd4530341c5b079e1752dcc0178 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178 2023-04-03 02:55:55,923 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178, entries=4, sequenceid=17, filesize=5.5 K 2023-04-03 02:55:55,924 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/75a78379469748d9b19b9584bf665bc7 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7 2023-04-03 02:55:55,927 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:55,928 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 672a7d65ccbb850591a80e29cf7d2b1b in 983ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:55,928 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:55,928 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:56,350 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304030a4287381919491fa94b14a9cd2ce010 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202304030a4287381919491fa94b14a9cd2ce010 2023-04-03 02:55:56,354 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/dae1525cf6ef486086adcb7933490b15 2023-04-03 02:55:56,358 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/a6b2d4aa1d4645d28805abcb793c08f8 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8 2023-04-03 02:55:56,382 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8, entries=4, sequenceid=17, filesize=5.5 K 2023-04-03 02:55:56,383 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/dae1525cf6ef486086adcb7933490b15 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15 2023-04-03 02:55:56,388 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15, entries=2, sequenceid=17, filesize=5.1 K 2023-04-03 02:55:56,389 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a8fff145c4646476fb737ed5f93141be in 1444ms, sequenceid=17, compaction requested=true 2023-04-03 02:55:56,389 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:56,389 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-03 02:55:56,389 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-03 02:55:56,391 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,391 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:56,391 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,391 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:56,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:56,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:56,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:56,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:56,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-03 02:55:56,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f63945c[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:56,393 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-03 02:55:56,393 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,394 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-03 02:55:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-03 02:55:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:56,396 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-03 02:55:56,396 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:56,396 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,396 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-03 02:55:56,397 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-03 02:55:56,397 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:56,397 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-03 02:55:56,397 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:56,397 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:56,397 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-03 02:55:56,397 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:56,399 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,399 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:56,399 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:56,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:56,400 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:56,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:56,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,402 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:56,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:56,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-03 02:55:56,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:56,404 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,404 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:56,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:56,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:56,404 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:56,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:56,405 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:56,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:56,505 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:56,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:56,516 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:56,523 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:56,531 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:55:56,532 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:55:56,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-03 02:55:56,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-03 02:55:56,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-03 02:55:56,623 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-03 02:55:56,623 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:56,623 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-03 02:55:56,623 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-03 02:55:56,623 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,623 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:56,624 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:56,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:56,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:56,624 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:56,625 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:56,625 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-03 02:55:56,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:56,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-03 02:55:56,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:56,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-03 02:55:56,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-03 02:55:56,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-03 02:55:56,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-03 02:55:56,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-03 02:55:56,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. started... 2023-04-03 02:55:56,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-03 02:55:56,626 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. started... 2023-04-03 02:55:56,626 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. started... 2023-04-03 02:55:56,626 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing a8fff145c4646476fb737ed5f93141be 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:56,626 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 66d14b50c01d6d8987769782f72db5e7 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:56,626 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 672a7d65ccbb850591a80e29cf7d2b1b 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-03 02:55:56,691 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/bf5666f4af9b46808bfbbbbf5edd3b15 2023-04-03 02:55:56,717 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5666f4af9b46808bfbbbbf5edd3b15 2023-04-03 02:55:57,082 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/47d154bb71ac4c328df904d1e506aa42 2023-04-03 02:55:57,106 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47d154bb71ac4c328df904d1e506aa42 2023-04-03 02:55:57,127 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/a1b01fca42e749619324edfef1c83194 2023-04-03 02:55:57,133 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a1b01fca42e749619324edfef1c83194 2023-04-03 02:55:57,134 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/47d154bb71ac4c328df904d1e506aa42 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42 2023-04-03 02:55:57,162 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 47d154bb71ac4c328df904d1e506aa42 2023-04-03 02:55:57,162 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42, entries=3, sequenceid=23, filesize=5.2 K 2023-04-03 02:55:57,163 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/a1b01fca42e749619324edfef1c83194 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194 2023-04-03 02:55:57,166 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a1b01fca42e749619324edfef1c83194 2023-04-03 02:55:57,166 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:57,167 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 672a7d65ccbb850591a80e29cf7d2b1b in 541ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:57,167 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:55:57,167 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:55:57,167 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-03 02:55:57,482 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/93d58ae7ae684c3fab6c14cb1c2209fb 2023-04-03 02:55:57,508 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93d58ae7ae684c3fab6c14cb1c2209fb 2023-04-03 02:55:57,534 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/85e70046c916483f8dc49f94926eff16 2023-04-03 02:55:57,538 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 85e70046c916483f8dc49f94926eff16 2023-04-03 02:55:57,538 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/bf5666f4af9b46808bfbbbbf5edd3b15 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15 2023-04-03 02:55:57,563 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5666f4af9b46808bfbbbbf5edd3b15 2023-04-03 02:55:57,564 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15, entries=3, sequenceid=23, filesize=5.2 K 2023-04-03 02:55:57,564 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/85e70046c916483f8dc49f94926eff16 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16 2023-04-03 02:55:57,569 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 85e70046c916483f8dc49f94926eff16 2023-04-03 02:55:57,569 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:57,570 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 66d14b50c01d6d8987769782f72db5e7 in 944ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:57,570 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:57,570 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:57,570 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-03 02:55:57,919 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/dbeae5caba724c69a2cb644bdd0b8831 2023-04-03 02:55:57,923 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dbeae5caba724c69a2cb644bdd0b8831 2023-04-03 02:55:57,924 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/93d58ae7ae684c3fab6c14cb1c2209fb as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb 2023-04-03 02:55:57,958 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93d58ae7ae684c3fab6c14cb1c2209fb 2023-04-03 02:55:57,958 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb, entries=3, sequenceid=23, filesize=5.2 K 2023-04-03 02:55:57,959 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/dbeae5caba724c69a2cb644bdd0b8831 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831 2023-04-03 02:55:57,963 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dbeae5caba724c69a2cb644bdd0b8831 2023-04-03 02:55:57,963 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831, entries=1, sequenceid=23, filesize=5.0 K 2023-04-03 02:55:57,964 INFO [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a8fff145c4646476fb737ed5f93141be in 1338ms, sequenceid=23, compaction requested=true 2023-04-03 02:55:57,964 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:55:57,964 DEBUG [rs(jenkins-hbase4.apache.org,45927,1680490420719)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:55:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-03 02:55:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-03 02:55:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-03 02:55:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-03 02:55:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,965 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:57,966 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,966 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-03 02:55:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:57,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:57,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45927,1680490420719' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-03 02:55:57,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f426630[Count = 0] remaining members to acquire global barrier 2023-04-03 02:55:57,968 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-03 02:55:57,968 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,969 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-03 02:55:57,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-03 02:55:57,969 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45927,1680490420719' in zk 2023-04-03 02:55:57,969 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,969 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-03 02:55:57,970 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,970 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-03 02:55:57,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:57,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:57,970 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:57,970 DEBUG [member: 'jenkins-hbase4.apache.org,45927,1680490420719' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-03 02:55:57,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:57,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:57,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:57,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45927,1680490420719': 2023-04-03 02:55:57,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45927,1680490420719' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-03 02:55:57,973 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-03 02:55:57,973 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-03 02:55:57,973 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-03 02:55:57,973 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-03 02:55:57,973 INFO [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-03 02:55:57,975 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,975 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-03 02:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-03 02:55:57,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,975 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:57,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-03 02:55:57,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,978 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-03 02:55:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,979 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-03 02:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-03 02:55:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-03 02:55:57,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-03 02:55:57,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-03 02:55:57,982 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-03 02:55:57,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-03 02:55:57,982 DEBUG [(jenkins-hbase4.apache.org,34259,1680490419507)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:55:57,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-03 02:55:57,983 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-03 02:55:57,982 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-03 02:55:57,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-03 02:55:57,983 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-03 02:55:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-03 02:55:58,083 DEBUG [Listener at localhost/37595] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-03 02:55:58,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34259] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-03 02:55:58,084 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:58,084 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 66d14b50c01d6d8987769782f72db5e7/family1 is initiating major compaction (all files) 2023-04-03 02:55:58,085 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:58,085 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:58,086 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 66d14b50c01d6d8987769782f72db5e7/family1 in testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:58,086 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp, totalSize=27.1 K 2023-04-03 02:55:58,086 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 7ebb4ecd6a9b4cd1b0ca26793bb442f0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549756 2023-04-03 02:55:58,087 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 23c047fd96e04d268ba9922bae60cfcb, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:55:58,087 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 7bcf502f1bc24a9092c2158fbe37c62d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:55:58,087 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 4f6b28ddb75c4613b41dcfadeb7ee9bb, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554933 2023-04-03 02:55:58,088 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting bf5666f4af9b46808bfbbbbf5edd3b15, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:58,098 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 66d14b50c01d6d8987769782f72db5e7#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:58,544 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del 2023-04-03 02:55:58,544 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del 2023-04-03 02:55:59,011 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24d2d7548e864d4585ad2290d5af76d0 2023-04-03 02:55:59,012 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family1/24d2d7548e864d4585ad2290d5af76d0 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/24d2d7548e864d4585ad2290d5af76d0 2023-04-03 02:55:59,039 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24d2d7548e864d4585ad2290d5af76d0 2023-04-03 02:55:59,040 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 66d14b50c01d6d8987769782f72db5e7/family1 of 66d14b50c01d6d8987769782f72db5e7 into 24d2d7548e864d4585ad2290d5af76d0(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:55:59,040 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:55:59,040 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:55:59,040 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 66d14b50c01d6d8987769782f72db5e7/family2 is initiating major compaction (all files) 2023-04-03 02:55:59,041 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:55:59,041 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:55:59,041 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 66d14b50c01d6d8987769782f72db5e7/family2 in testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:55:59,041 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp, totalSize=25.4 K 2023-04-03 02:55:59,041 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting edd83306bf0440fca60d891d36bf8a9d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549756 2023-04-03 02:55:59,042 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting c45e5c106b8144b1a55fa53e756ce6a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:55:59,043 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 83d2a0e0f00048fbaa78c2d8885a773b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:55:59,044 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 3af498a5620e4fa093a902109d649a04, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554933 2023-04-03 02:55:59,044 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 85e70046c916483f8dc49f94926eff16, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:55:59,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15] to archive 2023-04-03 02:55:59,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:55:59,058 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7ebb4ecd6a9b4cd1b0ca26793bb442f0 2023-04-03 02:55:59,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/23c047fd96e04d268ba9922bae60cfcb 2023-04-03 02:55:59,061 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/7bcf502f1bc24a9092c2158fbe37c62d 2023-04-03 02:55:59,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/4f6b28ddb75c4613b41dcfadeb7ee9bb 2023-04-03 02:55:59,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/bf5666f4af9b46808bfbbbbf5edd3b15 2023-04-03 02:55:59,068 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 66d14b50c01d6d8987769782f72db5e7#family2#compaction#198 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:55:59,486 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403a53a93f24a3b45a9a07d73f1adc6224e_del 2023-04-03 02:55:59,486 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230403a53a93f24a3b45a9a07d73f1adc6224e_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230403a53a93f24a3b45a9a07d73f1adc6224e_del 2023-04-03 02:56:00,299 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eba75d82dcb9457c9b0d4f05777f64e4 2023-04-03 02:56:00,299 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/.tmp/family2/eba75d82dcb9457c9b0d4f05777f64e4 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/eba75d82dcb9457c9b0d4f05777f64e4 2023-04-03 02:56:00,303 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eba75d82dcb9457c9b0d4f05777f64e4 2023-04-03 02:56:00,304 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 66d14b50c01d6d8987769782f72db5e7/family2 of 66d14b50c01d6d8987769782f72db5e7 into eba75d82dcb9457c9b0d4f05777f64e4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-03 02:56:00,304 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:56:00,304 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:56:00,304 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): a8fff145c4646476fb737ed5f93141be/family1 is initiating major compaction (all files) 2023-04-03 02:56:00,305 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:56:00,305 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:56:00,305 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of a8fff145c4646476fb737ed5f93141be/family1 in testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:00,305 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp, totalSize=27.1 K 2023-04-03 02:56:00,306 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 8fedeebe6f7e4de5a8b229b1a281fe43, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549756 2023-04-03 02:56:00,306 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 47ab455516bc4be6ab33b0236680f07a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:56:00,307 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 20fdd6df0a984880a75aabec561872ca, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:56:00,307 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting a6b2d4aa1d4645d28805abcb793c08f8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554932 2023-04-03 02:56:00,308 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 93d58ae7ae684c3fab6c14cb1c2209fb, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:56:00,319 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): a8fff145c4646476fb737ed5f93141be#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:56:00,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16] to archive 2023-04-03 02:56:00,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:00,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/edd83306bf0440fca60d891d36bf8a9d 2023-04-03 02:56:00,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/c45e5c106b8144b1a55fa53e756ce6a9 2023-04-03 02:56:00,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/83d2a0e0f00048fbaa78c2d8885a773b 2023-04-03 02:56:00,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/3af498a5620e4fa093a902109d649a04 2023-04-03 02:56:00,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family2/85e70046c916483f8dc49f94926eff16 2023-04-03 02:56:00,363 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del 2023-04-03 02:56:00,363 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del 2023-04-03 02:56:01,234 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc5059ab6e0b446dbdd5cf1796d044ea 2023-04-03 02:56:01,235 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family1/cc5059ab6e0b446dbdd5cf1796d044ea as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/cc5059ab6e0b446dbdd5cf1796d044ea 2023-04-03 02:56:01,259 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc5059ab6e0b446dbdd5cf1796d044ea 2023-04-03 02:56:01,259 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a8fff145c4646476fb737ed5f93141be/family1 of a8fff145c4646476fb737ed5f93141be into cc5059ab6e0b446dbdd5cf1796d044ea(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:56:01,260 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:56:01,260 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:56:01,260 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): a8fff145c4646476fb737ed5f93141be/family2 is initiating major compaction (all files) 2023-04-03 02:56:01,260 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:56:01,260 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:56:01,261 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of a8fff145c4646476fb737ed5f93141be/family2 in testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:01,261 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp, totalSize=25.4 K 2023-04-03 02:56:01,261 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting aca4566e080f4a2ebd90a9c108c2b016, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549756 2023-04-03 02:56:01,262 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 2e90b588ea2f4102b9b7da19e43592ca, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:56:01,262 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 0edceddd2fc04932b9d9d41f9c4ee3d2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:56:01,262 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting dae1525cf6ef486086adcb7933490b15, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554932 2023-04-03 02:56:01,263 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting dbeae5caba724c69a2cb644bdd0b8831, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:56:01,271 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): a8fff145c4646476fb737ed5f93141be#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:56:01,281 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023040394fca73f792943409bd15ce6f4b24fb8_del 2023-04-03 02:56:01,281 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023040394fca73f792943409bd15ce6f4b24fb8_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023040394fca73f792943409bd15ce6f4b24fb8_del 2023-04-03 02:56:01,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb] to archive 2023-04-03 02:56:01,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:01,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/8fedeebe6f7e4de5a8b229b1a281fe43 2023-04-03 02:56:01,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/47ab455516bc4be6ab33b0236680f07a 2023-04-03 02:56:01,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/20fdd6df0a984880a75aabec561872ca 2023-04-03 02:56:01,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/a6b2d4aa1d4645d28805abcb793c08f8 2023-04-03 02:56:01,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/93d58ae7ae684c3fab6c14cb1c2209fb 2023-04-03 02:56:01,694 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc88fa6e65b34039a954b92565542932 2023-04-03 02:56:01,695 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/.tmp/family2/dc88fa6e65b34039a954b92565542932 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dc88fa6e65b34039a954b92565542932 2023-04-03 02:56:01,699 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc88fa6e65b34039a954b92565542932 2023-04-03 02:56:01,700 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a8fff145c4646476fb737ed5f93141be/family2 of a8fff145c4646476fb737ed5f93141be into dc88fa6e65b34039a954b92565542932(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:56:01,700 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:56:01,700 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:56:01,700 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 672a7d65ccbb850591a80e29cf7d2b1b/family1 is initiating major compaction (all files) 2023-04-03 02:56:01,700 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:56:01,701 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:56:01,701 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 672a7d65ccbb850591a80e29cf7d2b1b/family1 in testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:01,701 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp, totalSize=27.1 K 2023-04-03 02:56:01,702 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting ef197984ab0f48cba90f9b0189b1b644, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549755 2023-04-03 02:56:01,702 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting d4aa62dd96a04a1bbc5ed42f61824fac, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:56:01,703 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting bba514dd5d9744caafaafd6314594a01, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:56:01,703 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 3e2f4cd4530341c5b079e1752dcc0178, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554932 2023-04-03 02:56:01,704 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 47d154bb71ac4c328df904d1e506aa42, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:56:01,713 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 672a7d65ccbb850591a80e29cf7d2b1b#family1#compaction#201 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:56:01,741 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831] to archive 2023-04-03 02:56:01,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:01,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/aca4566e080f4a2ebd90a9c108c2b016 2023-04-03 02:56:01,745 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/2e90b588ea2f4102b9b7da19e43592ca 2023-04-03 02:56:01,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/0edceddd2fc04932b9d9d41f9c4ee3d2 2023-04-03 02:56:01,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dae1525cf6ef486086adcb7933490b15 2023-04-03 02:56:01,749 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family2/dbeae5caba724c69a2cb644bdd0b8831 2023-04-03 02:56:01,752 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del 2023-04-03 02:56:01,753 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del 2023-04-03 02:56:01,818 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01d421c3d3724c6593c0dd0907ccd6fe 2023-04-03 02:56:01,819 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family1/01d421c3d3724c6593c0dd0907ccd6fe as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/01d421c3d3724c6593c0dd0907ccd6fe 2023-04-03 02:56:01,842 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01d421c3d3724c6593c0dd0907ccd6fe 2023-04-03 02:56:01,843 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 672a7d65ccbb850591a80e29cf7d2b1b/family1 of 672a7d65ccbb850591a80e29cf7d2b1b into 01d421c3d3724c6593c0dd0907ccd6fe(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:56:01,843 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:56:01,843 DEBUG [Listener at localhost/37595] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-03 02:56:01,843 DEBUG [Listener at localhost/37595] regionserver.HStore(1912): 672a7d65ccbb850591a80e29cf7d2b1b/family2 is initiating major compaction (all files) 2023-04-03 02:56:01,843 INFO [Listener at localhost/37595] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-03 02:56:01,843 INFO [Listener at localhost/37595] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-03 02:56:01,843 INFO [Listener at localhost/37595] regionserver.HRegion(2259): Starting compaction of 672a7d65ccbb850591a80e29cf7d2b1b/family2 in testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:01,843 INFO [Listener at localhost/37595] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194] into tmpdir=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp, totalSize=25.4 K 2023-04-03 02:56:01,844 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 4df7be20f0a9436384d89c26f3508cf0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1680490549755 2023-04-03 02:56:01,844 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 2c285546e0874788b33aa8fd6cb3e943, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1680490551615 2023-04-03 02:56:01,844 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 6e16033fdc2e48c3996235747be10232, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1680490553469 2023-04-03 02:56:01,845 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting 75a78379469748d9b19b9584bf665bc7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1680490554932 2023-04-03 02:56:01,845 DEBUG [Listener at localhost/37595] compactions.Compactor(207): Compacting a1b01fca42e749619324edfef1c83194, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-03 02:56:01,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42] to archive 2023-04-03 02:56:01,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:01,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/ef197984ab0f48cba90f9b0189b1b644 2023-04-03 02:56:01,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/d4aa62dd96a04a1bbc5ed42f61824fac 2023-04-03 02:56:01,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/bba514dd5d9744caafaafd6314594a01 2023-04-03 02:56:01,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/3e2f4cd4530341c5b079e1752dcc0178 2023-04-03 02:56:01,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/47d154bb71ac4c328df904d1e506aa42 2023-04-03 02:56:01,858 INFO [Listener at localhost/37595] throttle.PressureAwareThroughputController(145): 672a7d65ccbb850591a80e29cf7d2b1b#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-03 02:56:01,866 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230403ae4f453021a245cebd260df87c355f71_del 2023-04-03 02:56:01,866 INFO [Listener at localhost/37595] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230403ae4f453021a245cebd260df87c355f71_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230403ae4f453021a245cebd260df87c355f71_del 2023-04-03 02:56:01,877 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1fa2ea6bae01443e817c3e42f2c063e1 2023-04-03 02:56:01,877 DEBUG [Listener at localhost/37595] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/.tmp/family2/1fa2ea6bae01443e817c3e42f2c063e1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/1fa2ea6bae01443e817c3e42f2c063e1 2023-04-03 02:56:01,881 INFO [Listener at localhost/37595] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1fa2ea6bae01443e817c3e42f2c063e1 2023-04-03 02:56:01,882 INFO [Listener at localhost/37595] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 672a7d65ccbb850591a80e29cf7d2b1b/family2 of 672a7d65ccbb850591a80e29cf7d2b1b into 1fa2ea6bae01443e817c3e42f2c063e1(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-03 02:56:01,882 DEBUG [Listener at localhost/37595] regionserver.HRegion(2289): Compaction status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:56:01,887 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:56:01,890 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:56:01,895 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:56:01,897 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:56:01,902 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34259] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10g\x99\x8D\x8Eef\xE00\xC3\xA5\x03\x1B\x9B\xFE\xE6\xF7"\x10\x98\xBAap5\x99\xD3\x0E\x9E\xDB\x00\x1F\x5C\xE8Q\xEC*\x10J\xC2\xAC\xA8\xD3Z\x0A\xD2\x05AY\x0Du\xBF\xD6'2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-04-03 02:56:01,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-03 02:56:01,928 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-03 02:56:01,929 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:56:01,929 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1680491161929 2023-04-03 02:56:01,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-03 02:56:01,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194] to archive 2023-04-03 02:56:01,943 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:01,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/4df7be20f0a9436384d89c26f3508cf0 2023-04-03 02:56:01,948 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/2c285546e0874788b33aa8fd6cb3e943 2023-04-03 02:56:01,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/6e16033fdc2e48c3996235747be10232 2023-04-03 02:56:01,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/75a78379469748d9b19b9584bf665bc7 2023-04-03 02:56:01,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family2/a1b01fca42e749619324edfef1c83194 2023-04-03 02:56:02,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-04-03 02:56:02,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del 2023-04-03 02:56:02,427 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del 2023-04-03 02:56:02,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del 2023-04-03 02:56:02,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:02,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] zookeeper.ReadOnlyZKClient(139): Connect 0x3e97c14e to 127.0.0.1:52880 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-03 02:56:02,468 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1fbffb16, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-03 02:56:02,470 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:03,090 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040302facddce79d4b6b9b8d80cc07efe74e to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040302facddce79d4b6b9b8d80cc07efe74e 2023-04-03 02:56:03,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:03,108 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-03 02:56:03,112 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40898, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-03 02:56:03,137 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-03 02:56:03,140 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51170, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-03 02:56:03,181 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230403/family1/2f2b0e2b2ecd4c58a846f84b233598fd first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-04-03 02:56:03,182 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230403/family1/2f2b0e2b2ecd4c58a846f84b233598fd}] 2023-04-03 02:56:03,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230403/family1/2f2b0e2b2ecd4c58a846f84b233598fd for inclusion in 672a7d65ccbb850591a80e29cf7d2b1b/family1 2023-04-03 02:56:03,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-04-03 02:56:03,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first= last=1 2023-04-03 02:56:03,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:56:03,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230403/family1/2f2b0e2b2ecd4c58a846f84b233598fd to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__s64gvvmg4a7t5p2m07trlfh9l5h7g845t40cm9v2ag0c5e7ejo33rnsok2t1e2ij/family1/2f2b0e2b2ecd4c58a846f84b233598fd 2023-04-03 02:56:03,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__s64gvvmg4a7t5p2m07trlfh9l5h7g845t40cm9v2ag0c5e7ejo33rnsok2t1e2ij/family1/2f2b0e2b2ecd4c58a846f84b233598fd as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/27c463a38efa42f1be278cb558a74537_SeqId_28_ 2023-04-03 02:56:03,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__s64gvvmg4a7t5p2m07trlfh9l5h7g845t40cm9v2ag0c5e7ejo33rnsok2t1e2ij/family1/2f2b0e2b2ecd4c58a846f84b233598fd into 672a7d65ccbb850591a80e29cf7d2b1b/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/27c463a38efa42f1be278cb558a74537_SeqId_28_ - updating store file list. 2023-04-03 02:56:03,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/27c463a38efa42f1be278cb558a74537_SeqId_28_ into 672a7d65ccbb850591a80e29cf7d2b1b/family1 2023-04-03 02:56:03,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__s64gvvmg4a7t5p2m07trlfh9l5h7g845t40cm9v2ag0c5e7ejo33rnsok2t1e2ij/family1/2f2b0e2b2ecd4c58a846f84b233598fd into 672a7d65ccbb850591a80e29cf7d2b1b/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/family1/27c463a38efa42f1be278cb558a74537_SeqId_28_) 2023-04-03 02:56:03,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__s64gvvmg4a7t5p2m07trlfh9l5h7g845t40cm9v2ag0c5e7ejo33rnsok2t1e2ij/family1/2f2b0e2b2ecd4c58a846f84b233598fd 2023-04-03 02:56:03,252 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:03,253 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:03,255 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304035108c26f53204e80bd1a5c6285844328 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304035108c26f53204e80bd1a5c6285844328 2023-04-03 02:56:03,257 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037584cd86ced944178ace9c95cace18c0 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037584cd86ced944178ace9c95cace18c0 2023-04-03 02:56:03,258 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037c650f0304bb419a96fda88f00b7af38 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304037c650f0304bb419a96fda88f00b7af38 2023-04-03 02:56:03,259 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403df51cacde9494d159ce5454b1d739012 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403df51cacde9494d159ce5454b1d739012 2023-04-03 02:56:03,259 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-03 02:56:03,259 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230403' 2023-04-03 02:56:03,260 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:03,260 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:04,309 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040389d7ebe716fc48d5b33ce7baf255440c to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023040389d7ebe716fc48d5b33ce7baf255440c 2023-04-03 02:56:04,310 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:04,385 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230403/family1/7a2d739536fc4bc88492a98c7cb03e6d first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-04-03 02:56:04,386 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230403/family1/7a2d739536fc4bc88492a98c7cb03e6d}] 2023-04-03 02:56:04,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230403/family1/7a2d739536fc4bc88492a98c7cb03e6d for inclusion in 66d14b50c01d6d8987769782f72db5e7/family1 2023-04-03 02:56:04,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-04-03 02:56:04,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-03 02:56:04,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:56:04,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230403/family1/7a2d739536fc4bc88492a98c7cb03e6d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__o2km9dsr1vrhraj2hdmg27p0upb2fr239kg9gbbdebnbju5phn8jjp1ebmq4gnv3/family1/7a2d739536fc4bc88492a98c7cb03e6d 2023-04-03 02:56:04,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__o2km9dsr1vrhraj2hdmg27p0upb2fr239kg9gbbdebnbju5phn8jjp1ebmq4gnv3/family1/7a2d739536fc4bc88492a98c7cb03e6d as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/5ef7b5c92b684f47a6e43c43ec470ec6_SeqId_28_ 2023-04-03 02:56:04,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__o2km9dsr1vrhraj2hdmg27p0upb2fr239kg9gbbdebnbju5phn8jjp1ebmq4gnv3/family1/7a2d739536fc4bc88492a98c7cb03e6d into 66d14b50c01d6d8987769782f72db5e7/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/5ef7b5c92b684f47a6e43c43ec470ec6_SeqId_28_ - updating store file list. 2023-04-03 02:56:04,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/5ef7b5c92b684f47a6e43c43ec470ec6_SeqId_28_ into 66d14b50c01d6d8987769782f72db5e7/family1 2023-04-03 02:56:04,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__o2km9dsr1vrhraj2hdmg27p0upb2fr239kg9gbbdebnbju5phn8jjp1ebmq4gnv3/family1/7a2d739536fc4bc88492a98c7cb03e6d into 66d14b50c01d6d8987769782f72db5e7/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/family1/5ef7b5c92b684f47a6e43c43ec470ec6_SeqId_28_) 2023-04-03 02:56:04,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__o2km9dsr1vrhraj2hdmg27p0upb2fr239kg9gbbdebnbju5phn8jjp1ebmq4gnv3/family1/7a2d739536fc4bc88492a98c7cb03e6d 2023-04-03 02:56:04,461 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:04,462 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:04,463 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304034c8d75236c424ef3b0976c76d0ee095d to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304034c8d75236c424ef3b0976c76d0ee095d 2023-04-03 02:56:04,465 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304035f6d8a35cbc94260b041f46ea9095f19 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304035f6d8a35cbc94260b041f46ea9095f19 2023-04-03 02:56:04,466 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403852e69a5fff643d3abab0296b2e51978 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403852e69a5fff643d3abab0296b2e51978 2023-04-03 02:56:04,467 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403aac74ab4ec3646398577cc4f91725593 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403aac74ab4ec3646398577cc4f91725593 2023-04-03 02:56:04,467 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-03 02:56:04,467 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230403' 2023-04-03 02:56:04,467 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:04,467 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:05,042 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023040311b01e17142b45d9951098b87da50af7 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023040311b01e17142b45d9951098b87da50af7 2023-04-03 02:56:05,043 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:05,087 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230403/family1/f7edba256ec34519b9bb017addba1d69 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-04-03 02:56:05,088 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be., hostname=jenkins-hbase4.apache.org,45927,1680490420719, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230403/family1/f7edba256ec34519b9bb017addba1d69}] 2023-04-03 02:56:05,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(763): Validating hfile at hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230403/family1/f7edba256ec34519b9bb017addba1d69 for inclusion in a8fff145c4646476fb737ed5f93141be/family1 2023-04-03 02:56:05,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-04-03 02:56:05,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-03 02:56:05,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegion(2446): Flush status journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:56:05,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230403/family1/f7edba256ec34519b9bb017addba1d69 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__pndvehckdsalg2t57ov1klk4pdcgqku182j63k2knb6ngo4f8eittsgu7f9nhbj8/family1/f7edba256ec34519b9bb017addba1d69 2023-04-03 02:56:05,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__pndvehckdsalg2t57ov1klk4pdcgqku182j63k2knb6ngo4f8eittsgu7f9nhbj8/family1/f7edba256ec34519b9bb017addba1d69 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/969b4c6df7054b06a2388b3dbaf9fe12_SeqId_28_ 2023-04-03 02:56:05,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(849): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__pndvehckdsalg2t57ov1klk4pdcgqku182j63k2knb6ngo4f8eittsgu7f9nhbj8/family1/f7edba256ec34519b9bb017addba1d69 into a8fff145c4646476fb737ed5f93141be/family1 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/969b4c6df7054b06a2388b3dbaf9fe12_SeqId_28_ - updating store file list. 2023-04-03 02:56:05,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(882): Loaded HFile hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/969b4c6df7054b06a2388b3dbaf9fe12_SeqId_28_ into a8fff145c4646476fb737ed5f93141be/family1 2023-04-03 02:56:05,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.HStore(855): Successfully loaded hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__pndvehckdsalg2t57ov1klk4pdcgqku182j63k2knb6ngo4f8eittsgu7f9nhbj8/family1/f7edba256ec34519b9bb017addba1d69 into a8fff145c4646476fb737ed5f93141be/family1 (new location: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/family1/969b4c6df7054b06a2388b3dbaf9fe12_SeqId_28_) 2023-04-03 02:56:05,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45927] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/staging/jenkins__testMajorCompactionFromAdmin__pndvehckdsalg2t57ov1klk4pdcgqku182j63k2knb6ngo4f8eittsgu7f9nhbj8/family1/f7edba256ec34519b9bb017addba1d69 2023-04-03 02:56:05,151 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230403 table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:05,151 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:05,153 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304030b9a691d629548c4819b447d976a4fae to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304030b9a691d629548c4819b447d976a4fae 2023-04-03 02:56:05,154 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403604ed7e3e21e4f80b7530c38447e1263 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403604ed7e3e21e4f80b7530c38447e1263 2023-04-03 02:56:05,155 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304038f9c1546551b438a940c23f770f25a39 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304038f9c1546551b438a940c23f770f25a39 2023-04-03 02:56:05,156 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403a7cb7a69e94848cd858432b499891a00 to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230403a7cb7a69e94848cd858432b499891a00 2023-04-03 02:56:05,156 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-03 02:56:05,156 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230403' 2023-04-03 02:56:05,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230403 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:05,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-04-03 02:56:05,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-03 02:56:05,157 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del] 2023-04-03 02:56:05,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:05,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230403bdc839817d1d4fde9d8e72440146c04a_del 2023-04-03 02:56:05,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-03 02:56:05,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del] 2023-04-03 02:56:05,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:05,160 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023040345bd77590a544121bc9051c469f5f7fd_del 2023-04-03 02:56:05,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-03 02:56:05,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] compactions.PartitionedMobCompactor(367): [hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del] 2023-04-03 02:56:05,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(360): Archiving compacted files. 2023-04-03 02:56:05,162 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del to hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230403f7d3f7e816d245b480a0738e81b0aece_del 2023-04-03 02:56:05,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-03 02:56:05,162 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1680490424856] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-03 02:56:05,164 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-03 02:56:05,165 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2380 sec 2023-04-03 02:56:05,178 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:56:05,180 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:56:05,182 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-03 02:56:05,183 DEBUG [Listener at localhost/37595] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-03 02:56:05,190 INFO [Listener at localhost/37595] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-03 02:56:05,218 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:56:05,219 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e97c14e to 127.0.0.1:52880 2023-04-03 02:56:05,219 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,220 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:56:05,221 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25f72ce3 to 127.0.0.1:52880 2023-04-03 02:56:05,221 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,256 INFO [Listener at localhost/37595] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250 (was 262), OpenFileDescriptor=655 (was 647) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=442 (was 470), ProcessCount=170 (was 170), AvailableMemoryMB=3367 (was 3331) - AvailableMemoryMB LEAK? - 2023-04-03 02:56:05,256 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33fc1e18 to 127.0.0.1:52880 2023-04-03 02:56:05,257 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,257 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-04-03 02:56:05,257 INFO [Listener at localhost/37595] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-03 02:56:05,257 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0672b468 to 127.0.0.1:52880 2023-04-03 02:56:05,257 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,257 DEBUG [Listener at localhost/37595] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-04-03 02:56:05,258 DEBUG [Listener at localhost/37595] util.JVMClusterUtil(257): Found active master hash=337440889, stopped=false 2023-04-03 02:56:05,258 INFO [Listener at localhost/37595] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:56:05,261 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-03 02:56:05,261 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-03 02:56:05,261 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:56:05,261 INFO [Listener at localhost/37595] procedure2.ProcedureExecutor(629): Stopping 2023-04-03 02:56:05,261 DEBUG [Listener at localhost/37595] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7dd79577 to 127.0.0.1:52880 2023-04-03 02:56:05,261 DEBUG [Listener at localhost/37595] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-03 02:56:05,261 INFO [Listener at localhost/37595] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45927,1680490420719' ***** 2023-04-03 02:56:05,261 INFO [Listener at localhost/37595] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-03 02:56:05,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-03 02:56:05,267 INFO [RS:0;jenkins-hbase4:45927] regionserver.HeapMemoryManager(220): Stopping 2023-04-03 02:56:05,267 INFO [RS:0;jenkins-hbase4:45927] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-03 02:56:05,267 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-03 02:56:05,268 INFO [RS:0;jenkins-hbase4:45927] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-03 02:56:05,268 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 4d156a638f44baaa58723e6611f8ec49 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for dc22fb15c436508e6e6e9e67d65eb6d9 2023-04-03 02:56:05,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4d156a638f44baaa58723e6611f8ec49, disabling compactions & flushes 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for a431e83bfacaea3fe93968c2ff1d8e5a 2023-04-03 02:56:05,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 66d14b50c01d6d8987769782f72db5e7 2023-04-03 02:56:05,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 6b7f8c28a8619bf76181b65e5ae70d83 2023-04-03 02:56:05,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. after waiting 0 ms 2023-04-03 02:56:05,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 8d413c6a0cdb05564f66adc26a684a11 2023-04-03 02:56:05,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4d156a638f44baaa58723e6611f8ec49 1/1 column families, dataSize=109 B heapSize=592 B 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for a8fff145c4646476fb737ed5f93141be 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 1db75b42190724ca7b89184d5fdf0c8e 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 1af443f08637505427041327164c21b5 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 6e32a990bbbbb168b8d3ba4aabf4ebb2 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 7bcfadc80838c8916c1e93da8ae801f7 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 90ae7a86d67bedba5ae129851b513ac7 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 90d3745b2d090a6eb1d4604d211e6a26 2023-04-03 02:56:05,269 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 65d190e4c51d6a4a46cb459041dddd1e 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 91958613c4fcaae274fbfdf1b3c6a159 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 672a7d65ccbb850591a80e29cf7d2b1b 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 04487bf6cb13446fa4432d7bb8657941 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 5a1b3698dfeedaa8cd615262240ef68a 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 97be2f1adfec8336e4da255ab5f7cafd 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:56:05,270 DEBUG [RS:0;jenkins-hbase4:45927] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x502f0474 to 127.0.0.1:52880 2023-04-03 02:56:05,270 DEBUG [RS:0;jenkins-hbase4:45927] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,270 INFO [RS:0;jenkins-hbase4:45927] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-03 02:56:05,271 INFO [RS:0;jenkins-hbase4:45927] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-03 02:56:05,271 INFO [RS:0;jenkins-hbase4:45927] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-03 02:56:05,271 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-04-03 02:56:05,271 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-04-03 02:56:05,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-03 02:56:05,271 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-03 02:56:05,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-03 02:56:05,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-03 02:56:05,271 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-03 02:56:05,271 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-04-03 02:56:05,272 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1505): Waiting on 04487bf6cb13446fa4432d7bb8657941, 1588230740, 1af443f08637505427041327164c21b5, 1db75b42190724ca7b89184d5fdf0c8e, 4d156a638f44baaa58723e6611f8ec49, 5a1b3698dfeedaa8cd615262240ef68a, 65d190e4c51d6a4a46cb459041dddd1e, 66d14b50c01d6d8987769782f72db5e7, 672a7d65ccbb850591a80e29cf7d2b1b, 6b7f8c28a8619bf76181b65e5ae70d83, 6e32a990bbbbb168b8d3ba4aabf4ebb2, 7bcfadc80838c8916c1e93da8ae801f7, 8d413c6a0cdb05564f66adc26a684a11, 90ae7a86d67bedba5ae129851b513ac7, 90d3745b2d090a6eb1d4604d211e6a26, 91958613c4fcaae274fbfdf1b3c6a159, 97be2f1adfec8336e4da255ab5f7cafd, a431e83bfacaea3fe93968c2ff1d8e5a, a8fff145c4646476fb737ed5f93141be, dc22fb15c436508e6e6e9e67d65eb6d9, fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:56:05,280 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-03 02:56:05,288 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/info/418714e2f99d47d5bf15c03949582a4b 2023-04-03 02:56:05,292 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418714e2f99d47d5bf15c03949582a4b 2023-04-03 02:56:05,300 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/rep_barrier/cc85959856e043c287569e1250f698af 2023-04-03 02:56:05,304 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc85959856e043c287569e1250f698af 2023-04-03 02:56:05,311 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/table/b1e8ecafc0cc4396a539a766dc0a61b5 2023-04-03 02:56:05,313 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1e8ecafc0cc4396a539a766dc0a61b5 2023-04-03 02:56:05,314 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/info/418714e2f99d47d5bf15c03949582a4b as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/info/418714e2f99d47d5bf15c03949582a4b 2023-04-03 02:56:05,321 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418714e2f99d47d5bf15c03949582a4b 2023-04-03 02:56:05,321 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/info/418714e2f99d47d5bf15c03949582a4b, entries=226, sequenceid=130, filesize=37.9 K 2023-04-03 02:56:05,322 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/rep_barrier/cc85959856e043c287569e1250f698af as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/rep_barrier/cc85959856e043c287569e1250f698af 2023-04-03 02:56:05,325 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc85959856e043c287569e1250f698af 2023-04-03 02:56:05,325 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/rep_barrier/cc85959856e043c287569e1250f698af, entries=1, sequenceid=130, filesize=5.0 K 2023-04-03 02:56:05,326 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/.tmp/table/b1e8ecafc0cc4396a539a766dc0a61b5 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/table/b1e8ecafc0cc4396a539a766dc0a61b5 2023-04-03 02:56:05,329 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b1e8ecafc0cc4396a539a766dc0a61b5 2023-04-03 02:56:05,329 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/table/b1e8ecafc0cc4396a539a766dc0a61b5, entries=27, sequenceid=130, filesize=6.6 K 2023-04-03 02:56:05,330 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 59ms, sequenceid=130, compaction requested=false 2023-04-03 02:56:05,338 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-04-03 02:56:05,339 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-03 02:56:05,340 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-03 02:56:05,340 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-03 02:56:05,340 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-04-03 02:56:05,340 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-04-03 02:56:05,472 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1505): Waiting on 04487bf6cb13446fa4432d7bb8657941, 1af443f08637505427041327164c21b5, 1db75b42190724ca7b89184d5fdf0c8e, 4d156a638f44baaa58723e6611f8ec49, 5a1b3698dfeedaa8cd615262240ef68a, 65d190e4c51d6a4a46cb459041dddd1e, 66d14b50c01d6d8987769782f72db5e7, 672a7d65ccbb850591a80e29cf7d2b1b, 6b7f8c28a8619bf76181b65e5ae70d83, 6e32a990bbbbb168b8d3ba4aabf4ebb2, 7bcfadc80838c8916c1e93da8ae801f7, 8d413c6a0cdb05564f66adc26a684a11, 90ae7a86d67bedba5ae129851b513ac7, 90d3745b2d090a6eb1d4604d211e6a26, 91958613c4fcaae274fbfdf1b3c6a159, 97be2f1adfec8336e4da255ab5f7cafd, a431e83bfacaea3fe93968c2ff1d8e5a, a8fff145c4646476fb737ed5f93141be, dc22fb15c436508e6e6e9e67d65eb6d9, fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:56:05,672 DEBUG [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1505): Waiting on 04487bf6cb13446fa4432d7bb8657941, 1af443f08637505427041327164c21b5, 1db75b42190724ca7b89184d5fdf0c8e, 4d156a638f44baaa58723e6611f8ec49, 5a1b3698dfeedaa8cd615262240ef68a, 65d190e4c51d6a4a46cb459041dddd1e, 66d14b50c01d6d8987769782f72db5e7, 672a7d65ccbb850591a80e29cf7d2b1b, 6b7f8c28a8619bf76181b65e5ae70d83, 6e32a990bbbbb168b8d3ba4aabf4ebb2, 7bcfadc80838c8916c1e93da8ae801f7, 8d413c6a0cdb05564f66adc26a684a11, 90ae7a86d67bedba5ae129851b513ac7, 90d3745b2d090a6eb1d4604d211e6a26, 91958613c4fcaae274fbfdf1b3c6a159, 97be2f1adfec8336e4da255ab5f7cafd, a431e83bfacaea3fe93968c2ff1d8e5a, a8fff145c4646476fb737ed5f93141be, dc22fb15c436508e6e6e9e67d65eb6d9, fb728e0e72df91a59f2a870b06ac0762 2023-04-03 02:56:05,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/.tmp/info/910501af7f804805a7ebd8f780ea3f7e 2023-04-03 02:56:05,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/.tmp/info/910501af7f804805a7ebd8f780ea3f7e as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/info/910501af7f804805a7ebd8f780ea3f7e 2023-04-03 02:56:05,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/info/910501af7f804805a7ebd8f780ea3f7e, entries=3, sequenceid=7, filesize=4.8 K 2023-04-03 02:56:05,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 4d156a638f44baaa58723e6611f8ec49 in 418ms, sequenceid=7, compaction requested=false 2023-04-03 02:56:05,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/hbase/namespace/4d156a638f44baaa58723e6611f8ec49/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-04-03 02:56:05,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4d156a638f44baaa58723e6611f8ec49: 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1680490423338.4d156a638f44baaa58723e6611f8ec49. 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dc22fb15c436508e6e6e9e67d65eb6d9, disabling compactions & flushes 2023-04-03 02:56:05,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. after waiting 0 ms 2023-04-03 02:56:05,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:56:05,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/dc22fb15c436508e6e6e9e67d65eb6d9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dc22fb15c436508e6e6e9e67d65eb6d9: 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1680490520654.dc22fb15c436508e6e6e9e67d65eb6d9. 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a431e83bfacaea3fe93968c2ff1d8e5a, disabling compactions & flushes 2023-04-03 02:56:05,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. after waiting 0 ms 2023-04-03 02:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:56:05,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/a431e83bfacaea3fe93968c2ff1d8e5a/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-03 02:56:05,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a431e83bfacaea3fe93968c2ff1d8e5a: 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1680490500472.a431e83bfacaea3fe93968c2ff1d8e5a. 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 66d14b50c01d6d8987769782f72db5e7, disabling compactions & flushes 2023-04-03 02:56:05,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. after waiting 0 ms 2023-04-03 02:56:05,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:56:05,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/66d14b50c01d6d8987769782f72db5e7/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 66d14b50c01d6d8987769782f72db5e7: 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1680490547613.66d14b50c01d6d8987769782f72db5e7. 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6b7f8c28a8619bf76181b65e5ae70d83, disabling compactions & flushes 2023-04-03 02:56:05,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. after waiting 0 ms 2023-04-03 02:56:05,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:56:05,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/6b7f8c28a8619bf76181b65e5ae70d83/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-04-03 02:56:05,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6b7f8c28a8619bf76181b65e5ae70d83: 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1680490424953.6b7f8c28a8619bf76181b65e5ae70d83. 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8d413c6a0cdb05564f66adc26a684a11, disabling compactions & flushes 2023-04-03 02:56:05,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. after waiting 0 ms 2023-04-03 02:56:05,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:56:05,719 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyPolicy/8d413c6a0cdb05564f66adc26a684a11/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-03 02:56:05,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:56:05,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8d413c6a0cdb05564f66adc26a684a11: 2023-04-03 02:56:05,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1680490459905.8d413c6a0cdb05564f66adc26a684a11. 2023-04-03 02:56:05,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a8fff145c4646476fb737ed5f93141be, disabling compactions & flushes 2023-04-03 02:56:05,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:05,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:05,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. after waiting 0 ms 2023-04-03 02:56:05,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:05,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/a8fff145c4646476fb737ed5f93141be/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a8fff145c4646476fb737ed5f93141be: 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1680490547613.a8fff145c4646476fb737ed5f93141be. 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1db75b42190724ca7b89184d5fdf0c8e, disabling compactions & flushes 2023-04-03 02:56:05,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. after waiting 0 ms 2023-04-03 02:56:05,727 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:56:05,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/1db75b42190724ca7b89184d5fdf0c8e/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-03 02:56:05,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1db75b42190724ca7b89184d5fdf0c8e: 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1680490500472.1db75b42190724ca7b89184d5fdf0c8e. 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1af443f08637505427041327164c21b5, disabling compactions & flushes 2023-04-03 02:56:05,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. after waiting 0 ms 2023-04-03 02:56:05,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:56:05,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1af443f08637505427041327164c21b5/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-04-03 02:56:05,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1af443f08637505427041327164c21b5: 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1680490484050.1af443f08637505427041327164c21b5. 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6e32a990bbbbb168b8d3ba4aabf4ebb2, disabling compactions & flushes 2023-04-03 02:56:05,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. after waiting 0 ms 2023-04-03 02:56:05,736 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:56:05,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/6e32a990bbbbb168b8d3ba4aabf4ebb2/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6e32a990bbbbb168b8d3ba4aabf4ebb2: 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1680490520654.6e32a990bbbbb168b8d3ba4aabf4ebb2. 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7bcfadc80838c8916c1e93da8ae801f7, disabling compactions & flushes 2023-04-03 02:56:05,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. after waiting 0 ms 2023-04-03 02:56:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:56:05,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/7bcfadc80838c8916c1e93da8ae801f7/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-04-03 02:56:05,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7bcfadc80838c8916c1e93da8ae801f7: 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1680490456517.7bcfadc80838c8916c1e93da8ae801f7. 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 90ae7a86d67bedba5ae129851b513ac7, disabling compactions & flushes 2023-04-03 02:56:05,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. after waiting 0 ms 2023-04-03 02:56:05,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:56:05,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/90ae7a86d67bedba5ae129851b513ac7/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-04-03 02:56:05,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 90ae7a86d67bedba5ae129851b513ac7: 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1680490456517.90ae7a86d67bedba5ae129851b513ac7. 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 90d3745b2d090a6eb1d4604d211e6a26, disabling compactions & flushes 2023-04-03 02:56:05,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. after waiting 0 ms 2023-04-03 02:56:05,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:56:05,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/90d3745b2d090a6eb1d4604d211e6a26/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-04-03 02:56:05,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 90d3745b2d090a6eb1d4604d211e6a26: 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1680490469923.90d3745b2d090a6eb1d4604d211e6a26. 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 65d190e4c51d6a4a46cb459041dddd1e, disabling compactions & flushes 2023-04-03 02:56:05,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. after waiting 0 ms 2023-04-03 02:56:05,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:56:05,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testCompactionWithHFileLink/65d190e4c51d6a4a46cb459041dddd1e/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-03 02:56:05,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:56:05,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 65d190e4c51d6a4a46cb459041dddd1e: 2023-04-03 02:56:05,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1680490500472.65d190e4c51d6a4a46cb459041dddd1e. 2023-04-03 02:56:05,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 91958613c4fcaae274fbfdf1b3c6a159, disabling compactions & flushes 2023-04-03 02:56:05,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:56:05,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:56:05,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. after waiting 0 ms 2023-04-03 02:56:05,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:56:05,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithWeeklyPolicy/91958613c4fcaae274fbfdf1b3c6a159/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-04-03 02:56:05,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:56:05,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 91958613c4fcaae274fbfdf1b3c6a159: 2023-04-03 02:56:05,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1680490445630.91958613c4fcaae274fbfdf1b3c6a159. 2023-04-03 02:56:05,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 672a7d65ccbb850591a80e29cf7d2b1b, disabling compactions & flushes 2023-04-03 02:56:05,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:05,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:05,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. after waiting 0 ms 2023-04-03 02:56:05,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:05,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionFromAdmin/672a7d65ccbb850591a80e29cf7d2b1b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 672a7d65ccbb850591a80e29cf7d2b1b: 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1680490547613.672a7d65ccbb850591a80e29cf7d2b1b. 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 04487bf6cb13446fa4432d7bb8657941, disabling compactions & flushes 2023-04-03 02:56:05,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. after waiting 0 ms 2023-04-03 02:56:05,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:56:05,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMinorCompactionWithMonthlyPolicy/04487bf6cb13446fa4432d7bb8657941/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-04-03 02:56:05,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:56:05,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 04487bf6cb13446fa4432d7bb8657941: 2023-04-03 02:56:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1680490536141.04487bf6cb13446fa4432d7bb8657941. 2023-04-03 02:56:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5a1b3698dfeedaa8cd615262240ef68a, disabling compactions & flushes 2023-04-03 02:56:05,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:56:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:56:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. after waiting 0 ms 2023-04-03 02:56:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:56:05,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testMajorCompactionWithMonthlyPolicy/5a1b3698dfeedaa8cd615262240ef68a/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-04-03 02:56:05,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:56:05,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5a1b3698dfeedaa8cd615262240ef68a: 2023-04-03 02:56:05,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1680490437399.5a1b3698dfeedaa8cd615262240ef68a. 2023-04-03 02:56:05,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fb728e0e72df91a59f2a870b06ac0762, disabling compactions & flushes 2023-04-03 02:56:05,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:56:05,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:56:05,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. after waiting 0 ms 2023-04-03 02:56:05,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:56:05,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/ns/testMinorCompaction/fb728e0e72df91a59f2a870b06ac0762/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-03 02:56:05,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fb728e0e72df91a59f2a870b06ac0762: 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1680490520654.fb728e0e72df91a59f2a870b06ac0762. 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 97be2f1adfec8336e4da255ab5f7cafd, disabling compactions & flushes 2023-04-03 02:56:05,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. after waiting 0 ms 2023-04-03 02:56:05,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:56:05,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/data/default/testScannerOnBulkLoadRefHFiles/97be2f1adfec8336e4da255ab5f7cafd/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-04-03 02:56:05,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:56:05,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 97be2f1adfec8336e4da255ab5f7cafd: 2023-04-03 02:56:05,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1680490456517.97be2f1adfec8336e4da255ab5f7cafd. 2023-04-03 02:56:05,873 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45927,1680490420719; all regions closed. 2023-04-03 02:56:05,881 DEBUG [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/oldWALs 2023-04-03 02:56:05,881 INFO [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45927%2C1680490420719.meta:.meta(num 1680490423084) 2023-04-03 02:56:05,884 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/WALs/jenkins-hbase4.apache.org,45927,1680490420719/jenkins-hbase4.apache.org%2C45927%2C1680490420719.1680490422193 not finished, retry = 0 2023-04-03 02:56:05,987 DEBUG [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/oldWALs 2023-04-03 02:56:05,987 INFO [RS:0;jenkins-hbase4:45927] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45927%2C1680490420719:(num 1680490422193) 2023-04-03 02:56:05,987 DEBUG [RS:0;jenkins-hbase4:45927] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:05,987 INFO [RS:0;jenkins-hbase4:45927] regionserver.LeaseManager(133): Closed leases 2023-04-03 02:56:05,987 INFO [RS:0;jenkins-hbase4:45927] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-03 02:56:05,988 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-03 02:56:05,989 INFO [RS:0;jenkins-hbase4:45927] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45927 2023-04-03 02:56:05,997 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45927,1680490420719 2023-04-03 02:56:05,997 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-03 02:56:05,997 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-03 02:56:05,997 ERROR [Listener at localhost/37595-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a3f71be rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Shutting down, pool size = 1, active threads = 0, queued tasks = 1, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-03 02:56:05,998 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45927,1680490420719] 2023-04-03 02:56:05,998 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45927,1680490420719; numProcessing=1 2023-04-03 02:56:06,000 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45927,1680490420719 already deleted, retry=false 2023-04-03 02:56:06,000 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45927,1680490420719 expired; onlineServers=0 2023-04-03 02:56:06,000 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34259,1680490419507' ***** 2023-04-03 02:56:06,000 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-04-03 02:56:06,001 DEBUG [M:0;jenkins-hbase4:34259] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5521fa42, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-04-03 02:56:06,001 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34259,1680490419507 2023-04-03 02:56:06,001 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34259,1680490419507; all regions closed. 2023-04-03 02:56:06,001 DEBUG [M:0;jenkins-hbase4:34259] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-03 02:56:06,001 DEBUG [M:0;jenkins-hbase4:34259] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-04-03 02:56:06,001 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-04-03 02:56:06,002 DEBUG [M:0;jenkins-hbase4:34259] cleaner.HFileCleaner(317): Stopping file delete threads 2023-04-03 02:56:06,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1680490421819] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1680490421819,5,FailOnTimeoutGroup] 2023-04-03 02:56:06,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1680490421819] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1680490421819,5,FailOnTimeoutGroup] 2023-04-03 02:56:06,002 INFO [M:0;jenkins-hbase4:34259] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-04-03 02:56:06,002 INFO [M:0;jenkins-hbase4:34259] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-04-03 02:56:06,002 INFO [M:0;jenkins-hbase4:34259] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-04-03 02:56:06,003 DEBUG [M:0;jenkins-hbase4:34259] master.HMaster(1512): Stopping service threads 2023-04-03 02:56:06,003 INFO [M:0;jenkins-hbase4:34259] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-04-03 02:56:06,003 ERROR [M:0;jenkins-hbase4:34259] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-04-03 02:56:06,003 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-04-03 02:56:06,003 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-03 02:56:06,004 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-03 02:56:06,004 INFO [M:0;jenkins-hbase4:34259] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-03 02:56:06,004 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-04-03 02:56:06,004 DEBUG [M:0;jenkins-hbase4:34259] zookeeper.ZKUtil(398): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-04-03 02:56:06,004 WARN [M:0;jenkins-hbase4:34259] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-04-03 02:56:06,004 INFO [M:0;jenkins-hbase4:34259] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-03 02:56:06,005 INFO [M:0;jenkins-hbase4:34259] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-04-03 02:56:06,005 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-03 02:56:06,005 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:56:06,005 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:56:06,005 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-03 02:56:06,005 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:56:06,005 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.74 KB heapSize=598.41 KB 2023-04-03 02:56:06,017 INFO [M:0;jenkins-hbase4:34259] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.74 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6f9588b8008c44669d6dbbe935923e14 2023-04-03 02:56:06,022 INFO [M:0;jenkins-hbase4:34259] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6f9588b8008c44669d6dbbe935923e14 2023-04-03 02:56:06,022 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6f9588b8008c44669d6dbbe935923e14 as hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6f9588b8008c44669d6dbbe935923e14 2023-04-03 02:56:06,026 INFO [M:0;jenkins-hbase4:34259] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6f9588b8008c44669d6dbbe935923e14 2023-04-03 02:56:06,027 INFO [M:0;jenkins-hbase4:34259] regionserver.HStore(1080): Added hdfs://localhost:41571/user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6f9588b8008c44669d6dbbe935923e14, entries=123, sequenceid=1005, filesize=33.3 K 2023-04-03 02:56:06,027 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegion(2948): Finished flush of dataSize ~512.74 KB/525043, heapSize ~598.40 KB/612760, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1005, compaction requested=false 2023-04-03 02:56:06,028 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-03 02:56:06,028 DEBUG [M:0;jenkins-hbase4:34259] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-03 02:56:06,030 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/3da8fa32-1787-65fc-df69-6d6e78018e00/MasterData/WALs/jenkins-hbase4.apache.org,34259,1680490419507/jenkins-hbase4.apache.org%2C34259%2C1680490419507.1680490421177 not finished, retry = 0 2023-04-03 02:56:06,100 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-03 02:56:06,100 INFO [RS:0;jenkins-hbase4:45927] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45927,1680490420719; zookeeper connection closed. 2023-04-03 02:56:06,100 ERROR [Listener at localhost/37595-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@b4077a1 rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-03 02:56:06,100 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@26bad3db] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@26bad3db 2023-04-03 02:56:06,100 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): regionserver:45927-0x10147f0bb620001, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-03 02:56:06,100 ERROR [Listener at localhost/37595-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73bb176b rejected from java.util.concurrent.ThreadPoolExecutor@1e00f621[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-03 02:56:06,100 INFO [Listener at localhost/37595] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-04-03 02:56:06,131 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-03 02:56:06,131 INFO [M:0;jenkins-hbase4:34259] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-04-03 02:56:06,132 INFO [M:0;jenkins-hbase4:34259] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34259 2023-04-03 02:56:06,141 DEBUG [M:0;jenkins-hbase4:34259] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,34259,1680490419507 already deleted, retry=false 2023-04-03 02:56:06,242 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-03 02:56:06,242 INFO [M:0;jenkins-hbase4:34259] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34259,1680490419507; zookeeper connection closed. 2023-04-03 02:56:06,242 ERROR [Listener at localhost/37595-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@33f9ab32 rejected from java.util.concurrent.ThreadPoolExecutor@6bd9d4b3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 904] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-03 02:56:06,243 DEBUG [Listener at localhost/37595-EventThread] zookeeper.ZKWatcher(600): master:34259-0x10147f0bb620000, quorum=127.0.0.1:52880, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-03 02:56:06,243 ERROR [Listener at localhost/37595-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2bdabf11 rejected from java.util.concurrent.ThreadPoolExecutor@6bd9d4b3[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 904] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-03 02:56:06,245 WARN [Listener at localhost/37595] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-03 02:56:06,251 INFO [Listener at localhost/37595] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-03 02:56:06,355 WARN [BP-2002395145-172.31.14.131-1680490416057 heartbeating to localhost/127.0.0.1:41571] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-04-03 02:56:06,355 WARN [BP-2002395145-172.31.14.131-1680490416057 heartbeating to localhost/127.0.0.1:41571] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2002395145-172.31.14.131-1680490416057 (Datanode Uuid 1818b857-e78f-4c89-92f7-68f8617e3cc1) service to localhost/127.0.0.1:41571 2023-04-03 02:56:06,359 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/dfs/data/data1/current/BP-2002395145-172.31.14.131-1680490416057] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-03 02:56:06,360 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9c67d2a0-7c70-7119-c46d-ff0f62f86e3c/cluster_735a19f8-6410-716e-9bb7-2d72f83e61bc/dfs/data/data2/current/BP-2002395145-172.31.14.131-1680490416057] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-03 02:56:06,394 INFO [Listener at localhost/37595] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-03 02:56:06,440 INFO [Listener at localhost/37595] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-04-03 02:56:06,558 INFO [Listener at localhost/37595] hbase.HBaseTestingUtility(1293): Minicluster is down